Note that all users who use Vital DB, an open biosignal dataset, must agree to the Data Use Agreement below. If you do not agree, please close this window. The Data Use Agreement is available here: https://vitaldb.net/dataset/#h.vcpgs1yemdb5
For the Project Draft submission see the DL4H_Team_24_Project_Draft.ipynb notebook in the project repository.
The project repository can be found at: https://github.com/abarrie2/cs598-dlh-project
This project aims to reproduce findings from the paper titled "Predicting intraoperative hypotension using deep learning with waveforms of arterial blood pressure, electroencephalogram, and electrocardiogram: Retrospective study" by Jo Y-Y et al. (2022) [1]. This study introduces a deep learning model that predicts intraoperative hypotension (IOH) events before they occur, utilizing a combination of arterial blood pressure (ABP), electroencephalogram (EEG), and electrocardiogram (ECG) signals.
Intraoperative hypotension (IOH) is a common and significant surgical complication defined by a mean arterial pressure drop below 65 mmHg. It is associated with increased risks of myocardial infarction, acute kidney injury, and heightened postoperative mortality. Effective prediction and timely intervention can substantially enhance patient outcomes.
Initial attempts to predict IOH primarily used arterial blood pressure (ABP) waveforms. A foundational study by Hatib F et al. (2018) titled "Machine-learning Algorithm to Predict Hypotension Based on High-fidelity Arterial Pressure Waveform Analysis" [2] showed that machine learning could forecast IOH events using ABP with reasonable accuracy. This finding spurred further research into utilizing various physiological signals for IOH prediction.
Subsequent advancements included the development of the Acumen™ hypotension prediction index, which was studied in "AcumenTM hypotension prediction index guidance for prevention and treatment of hypotension in noncardiac surgery: a prospective, single-arm, multicenter trial" by Bao X et al. (2024) [3]. This trial integrated a hypotension prediction index into blood pressure monitoring equipment, demonstrating its effectiveness in reducing the number and duration of IOH events during surgeries. Further study is needed to determine whether this resultant reduction in IOH events transalates into improved postoperative patient outcomes.
Building on these advancements, the paper by Jo Y-Y et al. (2022) proposes a deep learning approach that enhances prediction accuracy by incorporating EEG and ECG signals along with ABP. This multi-modal method, evaluated over prediction windows of 3, 5, 10, and 15 minutes, aims to provide a comprehensive physiological profile that could predict IOH more accurately and earlier. Their results indicate that the combination of ABP and EEG significantly improves performance metrics such as AUROC and AUPRC, outperforming models that use fewer signals or different combinations.
Our project seeks to reproduce and verify Jo Y-Y et al.'s results to assess whether this integrated approach can indeed improve IOH prediction accuracy, thereby potentially enhancing surgical safety and patient outcomes.
The original paper investigated the following hypotheses:
Results were compared using AUROC and AUPRC scores. Based on the results described in the original paper, we expect that Hypothesis 2 will be confirmed, and that Hypotheses 1 and 3 will not be confirmed.
In order to perform the corresponding experiments, we will implement a CNN-based model that can be configured to train and infer using the following four model variations:
We will measure the performance of these configurations using the same AUROC and AUPRC metrics as used in the original paper. To test hypothesis 1 we will compare the AUROC and AUPRC measures between model variation 1 and model variation 2. To test hypothesis 2 we will compare the AUROC and AUPRC measures between model variation 1 and model variation 3. To test hypothesis 3 we will compare the AUROC and AUPRC measures between model variation 1 and model variation 4. For all of the above measures and experiment combinations, we will operate multiple experiments where the time-to-IOH event prediction will use the following prediction windows:
In the event that we are compute-bound, we will prioritize the 3-minute prediction window experiments as they are the most relevant to the original paper's findings.
The predictive power of ABP, ECG and ABP + ECG models at 3-, 5-, 10- and 15-minute prediction windows:
In order to demonstrate the functioning of the code in a short (ie, <8 minute limit) the following options and modifications were used:
MAX_CASES was set to 20. The total number of cases to be used in the full training set is 3296, but the smaller numbers allows demonstration of each section of the pipeline.vitaldb_cache is prepopulated in Google Colab. The cache file is approx. 800MB and contains the raw and mini-fied copies of the source dataset and is downloaded from Google Drive. This is much faster than using the vitaldb API, but is again only a fraction of the data. The full dataset can be downloaded with the API or prepopulated by following the instructions in the "Bulk Data Download" section below.max_epochs is set to 6. With the small dataset, training is fast and shows the decreasing training and validation losses. In the full model run, max_epochs will be set to 100. In both cases early stopping is enabled and will stop training if the validation losses stop decreasing for five consecutive epochs.The methodology section is composed of the following subsections: Environment, Data and Model.
The environment setup differs based on whether you are running the code on a local machine or on Google Colab. The following sections provide instructions for setting up the environment in each case.
Create conda environment for the project using the environment.yml file:
conda env create --prefix .envs/dlh-team24 -f environment.yml
Activate the environment with:
conda activate .envs/dlh-team24
The following code snippet installs the required packages and downloads the necessary files in a Google Colab environment:
# Google Colab environments have a `/content` directory. Use this as a proxy for running Colab-only code
COLAB_ENV = "google.colab" in str(get_ipython())
if COLAB_ENV:
#install vitaldb
%pip install vitaldb
# Executing in Colab therefore download cached preprocessed data.
# TODO: Integrate this with the setup local cache data section below.
# Check for file existence before overwriting.
import gdown
gdown.download(id="15b5Nfhgj3McSO2GmkVUKkhSSxQXX14hJ", output="vitaldb_cache.tgz")
!tar -zxf vitaldb_cache.tgz
# Download sqi_filter.csv from github repo
!wget https://raw.githubusercontent.com/abarrie2/cs598-dlh-project/main/sqi_filter.csv
All other required packages are already installed in the Google Colab environment.
# Import packages
import os
import random
import copy
from collections import defaultdict
from timeit import default_timer as timer
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from sklearn.manifold import TSNE
from sklearn.model_selection import train_test_split
from sklearn.metrics import classification_report, roc_auc_score, precision_recall_curve, auc, confusion_matrix
from sklearn.metrics import RocCurveDisplay, PrecisionRecallDisplay, average_precision_score
from sklearn.preprocessing import StandardScaler
from sklearn.neighbors import KNeighborsClassifier
import torch
from torch.utils.data import Dataset
import vitaldb
import h5py
import torch.nn as nn
import torch.nn.functional as F
from tqdm import tqdm
from datetime import datetime
Set random seeds to generate consistent results:
RANDOM_SEED = 42
random.seed(RANDOM_SEED)
np.random.seed(RANDOM_SEED)
torch.manual_seed(RANDOM_SEED)
if torch.cuda.is_available():
torch.cuda.manual_seed(RANDOM_SEED)
torch.cuda.manual_seed_all(RANDOM_SEED)
torch.backends.cudnn.deterministic = True
torch.backends.cudnn.benchmark = False
os.environ["PYTHONHASHSEED"] = str(RANDOM_SEED)
Set device to GPU or MPS if available
device = torch.device("cuda" if torch.cuda.is_available() else "mps" if (torch.backends.mps.is_available() and torch.backends.mps.is_built()) else "cpu")
print(f"Using device: {device}")
Using device: mps
Data for this project is sourced from the open biosignal VitalDB dataset as described in "VitalDB, a high-fidelity multi-parameter vital signs database in surgical patients" by Lee H-C et al. (2022) [4], which contains perioperative vital signs and numerical data from 6,388 cases of non-cardiac (general, thoracic, urological, and gynecological) surgery patients who underwent routine or emergency surgery at Seoul National University Hospital between 2016 and 2017. The dataset includes ABP, ECG, and EEG signals, as well as other physiological data. The dataset is available through an API and Python library, and at PhysioNet: https://physionet.org/content/vitaldb/1.0.0/
Characteristics of the dataset: | Characteristic | Value | Details | |-----------------------|-----------------------------|------------------------| | Total number of cases | 6,388 | | | Sex (male) | 3,243 (50.8%) | | | Age (years) | 59 | Range: 48-68 | | Height (cm) | 162 | Range: 156-169 | | Weight (kg) | 61 | Range: 53-69 | | Tram-Rac 4A tracks | 6,355 (99.5%) | Sampling rate: 500Hz | | BIS Vista tracks | 5,566 (87.1%) | Sampling rate: 128Hz | | Case duration (min) | 189 | Range: 27-1041 |
Labels are only known after processing the data. In the original paper, there were an average of 1.6 IOH events per case and 5.7 non-events per case so we expect approximately 10,221 IOH events and 364,116 non-events in the dataset.
Data will be processed as follows:
VitalDB data is static, so local copies can be stored and reused to avoid expensive downloads and to speed up data processing.
The default directory defined below is in the project .gitignore file. If this is modified, the new directory should also be added to the project .gitignore.
VITALDB_CACHE = './vitaldb_cache'
VITAL_ALL = f"{VITALDB_CACHE}/vital_all"
VITAL_MINI = f"{VITALDB_CACHE}/vital_mini"
VITAL_METADATA = f"{VITALDB_CACHE}/metadata"
VITAL_MODELS = f"{VITALDB_CACHE}/models"
VITAL_PREPROCESS_SCRATCH = f"{VITALDB_CACHE}/data_scratch"
VITAL_EXTRACTED_SEGMENTS = f"{VITALDB_CACHE}/segments"
TRACK_CACHE = None
SEGMENT_CACHE = None
# when USE_MEMORY_CACHING is enabled, track data will be persisted in an in-memory cache. Not useful once we have already pre-extracted all event segments
# DON'T USE: Stores items in memory that are later not used. Causes OOM on segment extraction.
USE_MEMORY_CACHING = False
# When RESET_CACHE is set to True, it will ensure the TRACK_CACHE is disposed and recreated when we do dataset initialization.
# Use as a shortcut to wiping cache rather than restarting kernel
RESET_CACHE = False
PREDICTION_WINDOW = 3
#PREDICTION_WINDOW = 'ALL'
ALL_PREDICTION_WINDOWS = [3, 5, 10, 15]
# Maximum number of cases of interest for which to download data.
# Set to a small value (ex: 20) for demo purposes, else set to None to disable and download and process all.
MAX_CASES = None
#MAX_CASES = 300
# Preloading Cases: when true, all matched cases will have the _mini tracks extracted and put into in-mem dict
PRELOADING_CASES = False
PRELOADING_SEGMENTS = True
# Perform Data Preprocessing: do we want to take the raw vital file and extract segments of interest for training?
PERFORM_DATA_PREPROCESSING = False
if not os.path.exists(VITALDB_CACHE):
os.mkdir(VITALDB_CACHE)
if not os.path.exists(VITAL_ALL):
os.mkdir(VITAL_ALL)
if not os.path.exists(VITAL_MINI):
os.mkdir(VITAL_MINI)
if not os.path.exists(VITAL_METADATA):
os.mkdir(VITAL_METADATA)
if not os.path.exists(VITAL_MODELS):
os.mkdir(VITAL_MODELS)
if not os.path.exists(VITAL_PREPROCESS_SCRATCH):
os.mkdir(VITAL_PREPROCESS_SCRATCH)
if not os.path.exists(VITAL_EXTRACTED_SEGMENTS):
os.mkdir(VITAL_EXTRACTED_SEGMENTS)
print(os.listdir(VITALDB_CACHE))
['segments_bak', '.DS_Store', 'vital_all', 'models_all_cases_baseline', 'models', 'docs', 'segments_bak_0501_00', 'vital_mini.tar', 'data_scratch', 'osfs', 'vital_mini', 'metadata', 'segments_bak_0428_00', 'segments', 'models_old']
This step is not required, but will significantly speed up downstream processing and avoid a high volume of API requests to the VitalDB web site.
The cache population code checks if the .vital files are locally available, and can be populated by calling the vitaldb API or by manually prepopulating the cache (recommended)
wget -r -N -c -np https://physionet.org/files/vitaldb/1.0.0/ to download the files in a terminalvital_files into the ${VITAL_ALL} directory.# Returns the Pandas DataFrame for the specified dataset.
# One of 'cases', 'labs', or 'trks'
# If the file exists locally, create and return the DataFrame.
# Else, download and cache the csv first, then return the DataFrame.
def vitaldb_dataframe_loader(dataset_name):
if dataset_name not in ['cases', 'labs', 'trks']:
raise ValueError(f'Invalid dataset name: {dataset_name}')
file_path = f'{VITAL_METADATA}/{dataset_name}.csv'
if os.path.isfile(file_path):
print(f'{dataset_name}.csv exists locally.')
df = pd.read_csv(file_path)
return df
else:
print(f'downloading {dataset_name} and storing in the local cache for future reuse.')
df = pd.read_csv(f'https://api.vitaldb.net/{dataset_name}')
df.to_csv(file_path, index=False)
return df
cases = vitaldb_dataframe_loader('cases')
cases = cases.set_index('caseid')
cases.shape
cases.csv exists locally.
(6388, 73)
cases.index.nunique()
6388
cases.head()
| subjectid | casestart | caseend | anestart | aneend | opstart | opend | adm | dis | icu_days | ... | intraop_colloid | intraop_ppf | intraop_mdz | intraop_ftn | intraop_rocu | intraop_vecu | intraop_eph | intraop_phe | intraop_epi | intraop_ca | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| caseid | |||||||||||||||||||||
| 1 | 5955 | 0 | 11542 | -552 | 10848.0 | 1668 | 10368 | -236220 | 627780 | 0 | ... | 0 | 120 | 0.0 | 100 | 70 | 0 | 10 | 0 | 0 | 0 |
| 2 | 2487 | 0 | 15741 | -1039 | 14921.0 | 1721 | 14621 | -221160 | 1506840 | 0 | ... | 0 | 150 | 0.0 | 0 | 100 | 0 | 20 | 0 | 0 | 0 |
| 3 | 2861 | 0 | 4394 | -590 | 4210.0 | 1090 | 3010 | -218640 | 40560 | 0 | ... | 0 | 0 | 0.0 | 0 | 50 | 0 | 0 | 0 | 0 | 0 |
| 4 | 1903 | 0 | 20990 | -778 | 20222.0 | 2522 | 17822 | -201120 | 576480 | 1 | ... | 0 | 80 | 0.0 | 100 | 100 | 0 | 50 | 0 | 0 | 0 |
| 5 | 4416 | 0 | 21531 | -1009 | 22391.0 | 2591 | 20291 | -67560 | 3734040 | 13 | ... | 0 | 0 | 0.0 | 0 | 160 | 0 | 10 | 900 | 0 | 2100 |
5 rows × 73 columns
cases['sex'].value_counts()
sex M 3243 F 3145 Name: count, dtype: int64
trks = vitaldb_dataframe_loader('trks')
trks = trks.set_index('caseid')
trks.shape
trks.csv exists locally.
(486449, 2)
trks.index.nunique()
6388
trks.groupby('caseid')[['tid']].count().plot();
trks.groupby('caseid')[['tid']].count().hist();
trks.groupby('tname').count().sort_values(by='tid', ascending=False)
| tid | |
|---|---|
| tname | |
| Solar8000/HR | 6387 |
| Solar8000/PLETH_SPO2 | 6386 |
| Solar8000/PLETH_HR | 6386 |
| Primus/CO2 | 6362 |
| Primus/PAMB_MBAR | 6361 |
| ... | ... |
| Orchestra/AMD_VOL | 1 |
| Solar8000/ST_V5 | 1 |
| Orchestra/NPS_VOL | 1 |
| Orchestra/AMD_RATE | 1 |
| Orchestra/VEC_VOL | 1 |
196 rows × 1 columns
SNUADC/ART
arterial blood pressure waveform
Parameter, Description, Type/Hz, Unit
SNUADC/ART, Arterial pressure wave, W/500, mmHg
trks[trks['tname'].str.contains('SNUADC/ART')].shape
(3645, 2)
SNUADC/ECG_II
electrocardiogram waveform
Parameter, Description, Type/Hz, Unit
SNUADC/ECG_II, ECG lead II wave, W/500, mV
trks[trks['tname'].str.contains('SNUADC/ECG_II')].shape
(6355, 2)
BIS/EEG1_WAV
electroencephalogram waveform
Parameter, Description, Type/Hz, Unit
BIS/EEG1_WAV, EEG wave from channel 1, W/128, uV
trks[trks['tname'].str.contains('BIS/EEG1_WAV')].shape
(5871, 2)
These are the subset of case ids for which modelling and analysis will be performed based upon inclusion criteria and waveform data availability.
# TRACK NAMES is used for metadata analysis via API
TRACK_NAMES = ['SNUADC/ART', 'SNUADC/ECG_II', 'BIS/EEG1_WAV']
TRACK_SRATES = [500, 500, 128]
# EXTRACTION TRACK NAMES adds the EVENT track which is only used when doing actual file i/o
EXTRACTION_TRACK_NAMES = ['SNUADC/ART', 'SNUADC/ECG_II', 'BIS/EEG1_WAV', 'EVENT']
EXTRACTION_TRACK_SRATES = [500, 500, 128, 1]
# As in the paper, select cases which meet the following criteria:
#
# For patients, the inclusion criteria were as follows:
# (1) adults (age >= 18)
# (2) administered general anaesthesia
# (3) undergone non-cardiac surgery.
#
# For waveform data, the inclusion criteria were as follows:
# (1) no missing monitoring for ABP, ECG, and EEG waveforms
# (2) no cases containing false events or non-events due to poor signal quality
# (checked in second stage of data preprocessing)
# Adult
inclusion_1 = cases.loc[cases['age'] >= 18].index
print(f'{len(cases)-len(inclusion_1)} cases excluded, {len(inclusion_1)} remaining due to age criteria')
# General Anesthesia
inclusion_2 = cases.loc[cases['ane_type'] == 'General'].index
print(f'{len(cases)-len(inclusion_2)} cases excluded, {len(inclusion_2)} remaining due to anesthesia criteria')
# Non-cardiac surgery
inclusion_3 = cases.loc[
~cases['opname'].str.contains("cardiac", case=False)
& ~cases['opname'].str.contains("aneurysmal", case=False)
].index
print(f'{len(cases)-len(inclusion_3)} cases excluded, {len(inclusion_3)} remaining due to non-cardiac surgery criteria')
# ABP, ECG, EEG waveforms
inclusion_4 = trks.loc[trks['tname'].isin(TRACK_NAMES)].index.value_counts()
inclusion_4 = inclusion_4[inclusion_4 == len(TRACK_NAMES)].index
print(f'{len(cases)-len(inclusion_4)} cases excluded, {len(inclusion_4)} remaining due to missing waveform data')
# SQI filter
# NOTE: this depends on a sqi_filter.csv generated by external processing
inclusion_5 = pd.read_csv('sqi_filter.csv', header=None, names=['caseid','sqi']).set_index('caseid').index
print(f'{len(cases)-len(inclusion_5)} cases excluded, {len(inclusion_5)} remaining due to SQI threshold not being met')
# Only include cases with known good waveforms.
exclusion_6 = pd.read_csv('malformed_tracks_filter.csv', header=None, names=['caseid']).set_index('caseid').index
inclusion_6 = cases.index.difference(exclusion_6)
print(f'{len(cases)-len(inclusion_6)} cases excluded, {len(inclusion_6)} remaining due to malformed waveforms')
cases_of_interest_idx = inclusion_1 \
.intersection(inclusion_2) \
.intersection(inclusion_3) \
.intersection(inclusion_4) \
.intersection(inclusion_5) \
.intersection(inclusion_6)
cases_of_interest = cases.loc[cases_of_interest_idx]
print()
print(f'{cases_of_interest_idx.shape[0]} out of {cases.shape[0]} total cases remaining after exclusions applied')
# Trim cases of interest to MAX_CASES
if MAX_CASES:
cases_of_interest_idx = cases_of_interest_idx[:MAX_CASES]
print(f'{cases_of_interest_idx.shape[0]} cases of interest selected')
57 cases excluded, 6331 remaining due to age criteria 345 cases excluded, 6043 remaining due to anesthesia criteria 14 cases excluded, 6374 remaining due to non-cardiac surgery criteria 3019 cases excluded, 3369 remaining due to missing waveform data 0 cases excluded, 6388 remaining due to SQI threshold not being met 186 cases excluded, 6202 remaining due to malformed waveforms 3110 out of 6388 total cases remaining after exclusions applied 3110 cases of interest selected
cases_of_interest.head(n=5)
| subjectid | casestart | caseend | anestart | aneend | opstart | opend | adm | dis | icu_days | ... | intraop_colloid | intraop_ppf | intraop_mdz | intraop_ftn | intraop_rocu | intraop_vecu | intraop_eph | intraop_phe | intraop_epi | intraop_ca | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| caseid | |||||||||||||||||||||
| 1 | 5955 | 0 | 11542 | -552 | 10848.0 | 1668 | 10368 | -236220 | 627780 | 0 | ... | 0 | 120 | 0.0 | 100 | 70 | 0 | 10 | 0 | 0 | 0 |
| 4 | 1903 | 0 | 20990 | -778 | 20222.0 | 2522 | 17822 | -201120 | 576480 | 1 | ... | 0 | 80 | 0.0 | 100 | 100 | 0 | 50 | 0 | 0 | 0 |
| 7 | 5124 | 0 | 15770 | 477 | 14817.0 | 3177 | 14577 | -154320 | 623280 | 3 | ... | 0 | 0 | 0.0 | 0 | 120 | 0 | 0 | 0 | 0 | 0 |
| 10 | 2175 | 0 | 20992 | -1743 | 21057.0 | 2457 | 19857 | -220740 | 3580860 | 1 | ... | 0 | 90 | 0.0 | 0 | 110 | 0 | 20 | 500 | 0 | 600 |
| 12 | 491 | 0 | 31203 | -220 | 31460.0 | 5360 | 30860 | -208500 | 1519500 | 4 | ... | 200 | 100 | 0.0 | 100 | 70 | 0 | 20 | 0 | 0 | 3300 |
5 rows × 73 columns
These are the subset of tracks (waveforms) for the cases of interest identified above.
# A single case maps to one or more waveform tracks. Select only the tracks required for analysis.
trks_of_interest = trks.loc[cases_of_interest_idx][trks.loc[cases_of_interest_idx]['tname'].isin(TRACK_NAMES)]
trks_of_interest.shape
(9330, 2)
trks_of_interest.head(n=5)
| tname | tid | |
|---|---|---|
| caseid | ||
| 1 | BIS/EEG1_WAV | 0aa685df768489a18a5e9f53af0d83bf60890c73 |
| 1 | SNUADC/ART | 724cdd7184d7886b8f7de091c5b135bd01949959 |
| 1 | SNUADC/ECG_II | 8c9161aaae8cb578e2aa7b60f44234d98d2b3344 |
| 4 | BIS/EEG1_WAV | 1b4c2379be3397a79d3787dd810190150dc53f27 |
| 4 | SNUADC/ART | e28777c4706fe3a5e714bf2d91821d22d782d802 |
trks_of_interest_idx = trks_of_interest.set_index('tid').index
trks_of_interest_idx.shape
(9330,)
Tracks data are large and therefore expensive to download every time used.
By default, the .vital file format stores all tracks for each case internally. Since only select tracks per case are required, each .vital file can be further reduced by discarding the unused tracks.
# Ensure the full vital file dataset is available for cases of interest.
count_downloaded = 0
count_present = 0
#for i, idx in enumerate(cases.index):
for idx in cases_of_interest_idx:
full_path = f'{VITAL_ALL}/{idx:04d}.vital'
if not os.path.isfile(full_path):
print(f'Missing vital file: {full_path}')
# Download and save the file.
vf = vitaldb.VitalFile(idx)
vf.to_vital(full_path)
count_downloaded += 1
else:
count_present += 1
print()
print(f'Count of cases of interest: {cases_of_interest_idx.shape[0]}')
print(f'Count of vital files downloaded: {count_downloaded}')
print(f'Count of vital files already present: {count_present}')
Count of cases of interest: 3110 Count of vital files downloaded: 0 Count of vital files already present: 3110
# Convert vital files to "mini" versions including only the subset of tracks defined in TRACK_NAMES above.
# Only perform conversion for the cases of interest.
# NOTE: If this cell is interrupted, it can be restarted and will continue where it left off.
count_minified = 0
count_present = 0
count_missing_tracks = 0
count_not_fixable = 0
vf = vitaldb.VitalFile('./vitaldb_cache/vital_all/0001.vital', EXTRACTION_TRACK_NAMES)
print(vf)
# If set to true, local mini files are checked for all tracks even if already present.
FORCE_VALIDATE = False
for idx in cases_of_interest_idx:
full_path = f'{VITAL_ALL}/{idx:04d}.vital'
mini_path = f'{VITAL_MINI}/{idx:04d}_mini.vital'
if FORCE_VALIDATE or not os.path.isfile(mini_path):
print(f'Creating mini vital file: {idx}')
vf = vitaldb.VitalFile(full_path, EXTRACTION_TRACK_NAMES)
if len(vf.get_track_names()) != 4:
print(f'Missing track in vital file: {idx}, {set(EXTRACTION_TRACK_NAMES).difference(set(vf.get_track_names()))}')
count_missing_tracks += 1
# Attempt to download from VitalDB directly and see if missing tracks are present.
vf = vitaldb.VitalFile(idx, EXTRACTION_TRACK_NAMES)
if len(vf.get_track_names()) != 3:
print(f'Unable to fix missing tracks: {idx}')
count_not_fixable += 1
continue
if vf.get_track_samples(EXTRACTION_TRACK_NAMES[0], 1/EXTRACTION_TRACK_SRATES[0]).shape[0] == 0:
print(f'Empty track: {idx}, {EXTRACTION_TRACK_NAMES[0]}')
count_not_fixable += 1
continue
if vf.get_track_samples(EXTRACTION_TRACK_NAMES[1], 1/EXTRACTION_TRACK_SRATES[1]).shape[0] == 0:
print(f'Empty track: {idx}, {EXTRACTION_TRACK_NAMES[1]}')
count_not_fixable += 1
continue
if vf.get_track_samples(EXTRACTION_TRACK_NAMES[2], 1/EXTRACTION_TRACK_SRATES[2]).shape[0] == 0:
print(f'Empty track: {idx}, {EXTRACTION_TRACK_NAMES[2]}')
count_not_fixable += 1
continue
# if vf.get_track_samples(EXTRACTION_TRACK_NAMES[3], 1/EXTRACTION_TRACK_SRATES[3]).shape[0] == 0:
# print(f'Empty track: {idx}, {EXTRACTION_TRACK_NAMES[3]}')
# count_not_fixable += 1
# continue
vf.to_vital(mini_path)
count_minified += 1
else:
count_present += 1
print()
print(f'Count of cases of interest: {cases_of_interest_idx.shape[0]}')
print(f'Count of vital files minified: {count_minified}')
print(f'Count of vital files already present: {count_present}')
print(f'Count of vital files missing tracks: {count_missing_tracks}')
print(f'Count of vital files not fixable: {count_not_fixable}')
VitalFile('./vitaldb_cache/vital_all/0001.vital', '['EVENT', 'SNUADC/ART', 'SNUADC/ECG_II', 'BIS/EEG1_WAV']')
Count of cases of interest: 3110
Count of vital files minified: 0
Count of vital files already present: 3110
Count of vital files missing tracks: 0
Count of vital files not fixable: 0
# Convert vital files to "mini" versions including only the subset of tracks defined in TRACK_NAMES above.
# Only perform conversion for the cases of interest.
# NOTE: If this cell is interrupted, it can be restarted and will continue where it left off.
count_missing_tracks = 0
# If true, perform fast validate that all mini files have 3 tracks.
FORCE_VALIDATE = False
if FORCE_VALIDATE:
for idx in cases_of_interest_idx:
mini_path = f'{VITAL_MINI}/{idx:04d}_mini.vital'
if os.path.isfile(mini_path):
vf = vitaldb.VitalFile(mini_path)
if len(vf.get_track_names()) != 3:
print(f'Missing track in vital file: {idx}, {set(TRACK_NAMES).difference(set(vf.get_track_names()))}')
count_missing_tracks += 1
print()
print(f'Count of cases of interest: {cases_of_interest_idx.shape[0]}')
print(f'Count of vital files missing tracks: {count_missing_tracks}')
Count of cases of interest: 3110 Count of vital files missing tracks: 0
Preprocessing characteristics are different for each of the three signal categories:
apply_bandpass_filter() implements the bandpass filter using scipy.signal
apply_zscore_normalization() implements the Z-score normalization using numpy
from scipy.signal import butter, lfilter, spectrogram
# define two methods for data preprocessing
def apply_bandpass_filter(data, lowcut, highcut, fs, order=5):
b, a = butter(order, [lowcut, highcut], fs=fs, btype='band')
y = lfilter(b, a, np.nan_to_num(data))
return y
def apply_zscore_normalization(signal):
mean = np.nanmean(signal)
std = np.nanstd(signal)
return (signal - mean) / std
# Filtering Demonstration
# temp experimental, code to be incorporated into overall preloader process
# for now it's just dumping example plots of the before/after filtered signal data
caseidx = 1
file_path = f"{VITAL_MINI}/{caseidx:04d}_mini.vital"
vf = vitaldb.VitalFile(file_path, TRACK_NAMES)
originalAbp = None
filteredAbp = None
originalEcg = None
filteredEcg = None
originalEeg = None
filteredEeg = None
ABP_TRACK_NAME = "SNUADC/ART"
ECG_TRACK_NAME = "SNUADC/ECG_II"
EEG_TRACK_NAME = "BIS/EEG1_WAV"
for i, (track_name, rate) in enumerate(zip(TRACK_NAMES, TRACK_SRATES)):
# Get samples for this track
track_samples = vf.get_track_samples(track_name, 1/rate)
#track_samples, _ = vf.get_samples(track_name, 1/rate)
print(f"Track {track_name} @ {rate}Hz shape {len(track_samples)}")
if track_name == ABP_TRACK_NAME:
# ABP waveforms are used without further pre-processing
originalAbp = track_samples
filteredAbp = track_samples
elif track_name == ECG_TRACK_NAME:
originalEcg = track_samples
# ECG waveforms are band-pass filtered between 1 and 40 Hz, and Z-score normalized
# first apply bandpass filter
filteredEcg = apply_bandpass_filter(track_samples, 1, 40, rate)
# then do z-score normalization
filteredEcg = apply_zscore_normalization(filteredEcg)
elif track_name == EEG_TRACK_NAME:
# EEG waveforms are band-pass filtered between 0.5 and 50 Hz
originalEeg = track_samples
filteredEeg = apply_bandpass_filter(track_samples, 0.5, 50, rate, 2)
def plotSignal(data, title):
plt.figure(figsize=(20, 5))
plt.plot(data)
plt.title(title)
plt.show()
plotSignal(originalAbp, "Original ABP")
plotSignal(originalAbp, "Unfiltered ABP")
plotSignal(originalEcg, "Original ECG")
plotSignal(filteredEcg, "Filtered ECG")
plotSignal(originalEeg, "Original EEG")
plotSignal(filteredEeg, "Filtered EEG")
Track SNUADC/ART @ 500Hz shape 5770575 Track SNUADC/ECG_II @ 500Hz shape 5770575 Track BIS/EEG1_WAV @ 128Hz shape 1477268
# Preprocess data tracks
ABP_TRACK_NAME = "SNUADC/ART"
ECG_TRACK_NAME = "SNUADC/ECG_II"
EEG_TRACK_NAME = "BIS/EEG1_WAV"
EVENT_TRACK_NAME = "EVENT"
MINI_FILE_FOLDER = VITAL_MINI
CACHE_FILE_FOLDER = VITAL_PREPROCESS_SCRATCH
if RESET_CACHE:
TRACK_CACHE = None
SEGMENT_CACHE = None
if TRACK_CACHE is None:
TRACK_CACHE = {}
SEGMENT_CACHE = {}
def get_track_data(case, print_when_file_loaded = False):
parsedFile = None
abp = None
eeg = None
ecg = None
events = None
for i, (track_name, rate) in enumerate(zip(EXTRACTION_TRACK_NAMES, EXTRACTION_TRACK_SRATES)):
# use integer case id and track name, delimited by pipe, as cache key
cache_label = f"{case}|{track_name}"
if cache_label not in TRACK_CACHE:
if parsedFile is None:
file_path = f"{MINI_FILE_FOLDER}/{case:04d}_mini.vital"
if print_when_file_loaded:
print(f"[{datetime.now()}] Loading vital file {file_path}")
parsedFile = vitaldb.VitalFile(file_path, EXTRACTION_TRACK_NAMES)
dataset = np.array(parsedFile.get_track_samples(track_name, 1/rate))
if track_name == ABP_TRACK_NAME:
# no filtering for ABP
abp = dataset
abp = pd.DataFrame(abp).ffill(axis=0).bfill(axis=0)[0].values
if USE_MEMORY_CACHING:
TRACK_CACHE[cache_label] = abp
elif track_name == ECG_TRACK_NAME:
ecg = dataset
# apply ECG filtering: first bandpass then do z-score normalization
ecg = pd.DataFrame(ecg).ffill(axis=0).bfill(axis=0)[0].values
ecg = apply_bandpass_filter(ecg, 1, 40, rate, 2)
ecg = apply_zscore_normalization(ecg)
if USE_MEMORY_CACHING:
TRACK_CACHE[cache_label] = ecg
elif track_name == EEG_TRACK_NAME:
eeg = dataset
eeg = pd.DataFrame(eeg).ffill(axis=0).bfill(axis=0)[0].values
# apply EEG filtering: bandpass only
eeg = apply_bandpass_filter(eeg, 0.5, 50, rate, 2)
if USE_MEMORY_CACHING:
TRACK_CACHE[cache_label] = eeg
elif track_name == EVENT_TRACK_NAME:
events = dataset
if USE_MEMORY_CACHING:
TRACK_CACHE[cache_label] = events
else:
# cache hit, pull from cache
if track_name == ABP_TRACK_NAME:
abp = TRACK_CACHE[cache_label]
elif track_name == ECG_TRACK_NAME:
ecg = TRACK_CACHE[cache_label]
elif track_name == EEG_TRACK_NAME:
eeg = TRACK_CACHE[cache_label]
elif track_name == EVENT_TRACK_NAME:
events = TRACK_CACHE[cache_label]
return (abp, ecg, eeg, events)
# ABP waveforms are used without further pre-processing
# ECG waveforms are band-pass filtered between 1 and 40 Hz, and Z-score normalized
# EEG waveforms are band-pass filtered between 0.5 and 50 Hz
if PRELOADING_CASES:
# determine disk cache file label
maxlabel = "ALL"
if MAX_CASES is not None:
maxlabel = str(MAX_CASES)
picklefile = f"{CACHE_FILE_FOLDER}/{PREDICTION_WINDOW}_minutes_MAX{maxlabel}.trackcache"
for track in tqdm(cases_of_interest_idx):
# getting track data will cause a cache-check and fill when missing
# will also apply appropriate filtering per track
get_track_data(track, False)
print(f"Generated track cache, {len(TRACK_CACHE)} records generated")
def get_segment_data(file_path):
abp = None
eeg = None
ecg = None
if USE_MEMORY_CACHING:
if file_path in SEGMENT_CACHE:
(abp, ecg, eeg) = SEGMENT_CACHE[file_path]
return (abp, ecg, eeg)
try:
with h5py.File(file_path, 'r') as f:
abp = np.array(f['abp'])
ecg = np.array(f['ecg'])
eeg = np.array(f['eeg'])
abp = np.array(abp)
eeg = np.array(eeg)
ecg = np.array(ecg)
if len(abp) > 30000:
abp = abp[:30000]
elif len(ecg) < 30000:
abp = np.resize(abp, (30000))
if len(ecg) > 30000:
ecg = ecg[:30000]
elif len(ecg) < 30000:
ecg = np.resize(ecg, (30000))
if len(eeg) > 7680:
eeg = eeg[:7680]
elif len(eeg) < 7680:
eeg = np.resize(eeg, (7680))
if USE_MEMORY_CACHING:
SEGMENT_CACHE[file_path] = (abp, ecg, eeg)
except:
abp = None
ecg = None
eeg = None
return (abp, ecg, eeg)
The following method is adapted from the preprocessing block of reference [6] (https://github.com/vitaldb/examples/blob/master/hypotension_art.ipynb)
The approach first finds an interoperative hypotensive event in the ABP waveform. It then backtracks to earlier in the waveform to extract a 60 second segment representing the waveform feature to use as model input. The figure below shows an example of this approach and is reproduced from the VitalDB example notebook referenced above.

def getSurgeryBoundariesInSeconds(event, debug=False):
eventIndices = np.argwhere(event==event)
# we are looking for the last index where the string contains 'start
lastStart = 0
firstFinish = len(event)-1
# find last start
for idx in eventIndices:
if 'started' in event[idx[0]]:
if debug:
print(event[idx[0]])
print(idx[0])
lastStart = idx[0]
# find first finish
for idx in eventIndices:
if 'finish' in event[idx[0]]:
if debug:
print(event[idx[0]])
print(idx[0])
firstFinish = idx[0]
break
if debug:
print(f'lastStart, firstFinish: {lastStart}, {firstFinish}')
return (lastStart, firstFinish)
def areCaseSegmentsCached(caseid):
seg_folder = f"{VITAL_EXTRACTED_SEGMENTS}/{caseid:04d}"
return os.path.exists(seg_folder) and len(os.listdir(seg_folder)) > 0
def isAbpSegmentValidNumpy(samples, debug=False):
valid = True
if np.isnan(samples).mean() > 0.1:
valid = False
if debug:
print(f">10% NaN")
elif (samples > 200).any():
valid = False
if debug:
print(f"Presence of BP > 200")
elif (samples < 30).any():
valid = False
if debug:
print(f"Presence of BP < 30")
elif np.max(samples) - np.min(samples) < 30:
if debug:
print(f"Max - Min test < 30")
valid = False
elif (np.abs(np.diff(samples)) > 30).any(): # abrupt change -> noise
if debug:
print(f"Abrupt change (noise)")
valid = False
return valid
def isAbpSegmentValid(vf, debug=False):
ABP_ECG_SRATE_HZ = 500
ABP_TRACK_NAME = "SNUADC/ART"
samples = np.array(vf.get_track_samples(ABP_TRACK_NAME, 1/ABP_ECG_SRATE_HZ))
return isAbpSegmentValidNumpy(samples, debug)
def saveCaseSegments(caseid, positiveSegments, negativeSegments, compresslevel=9, debug=False, forceWrite=False):
if len(positiveSegments) == 0 and len(negativeSegments) == 0:
# exit early if no events found
print(f'{caseid}: exit early, no segments to save')
return
# event composition
# predictiveSegmentStart in seconds, predictiveSegmentEnd in seconds, predWindow (0 for negative), abp, ecg, eeg)
# 0start, 1end, 2predwindow, 3abp, 4ecg, 5eeg
seg_folder = f"{VITAL_EXTRACTED_SEGMENTS}/{caseid:04d}"
if not os.path.exists(seg_folder):
# if directory needs to be created, then there are no cached segments
os.mkdir(seg_folder)
else:
if not forceWrite:
# exit early if folder already exists, case already produced
return
# prior to writing files out, clear existing files
for filename in os.listdir(seg_folder):
file_path = os.path.join(seg_folder, filename)
if debug:
print(f'deleting: {file_path}')
try:
if os.path.isfile(file_path):
os.unlink(file_path)
except Exception as e:
print('Failed to delete %s. Reason: %s' % (file_path, e))
count_pos_saved = 0
for i in range(0, len(positiveSegments)):
event = positiveSegments[i]
startIndex = event[0]
endIndex = event[1]
predWindow = event[2]
abp = event[3]
#ecg = event[4]
#eeg = event[5]
seg_filename = f"{caseid:04d}_{startIndex}_{predWindow:02d}_True.h5"
seg_fullpath = f"{seg_folder}/{seg_filename}"
if isAbpSegmentValidNumpy(abp, debug):
count_pos_saved += 1
abp = abp.tolist()
ecg = event[4].tolist()
eeg = event[5].tolist()
f = h5py.File(seg_fullpath, "w")
f.create_dataset('abp', data=abp, compression="gzip", compression_opts=compresslevel)
f.create_dataset('ecg', data=ecg, compression="gzip", compression_opts=compresslevel)
f.create_dataset('eeg', data=eeg, compression="gzip", compression_opts=compresslevel)
f.flush()
f.close()
f = None
abp = None
ecg = None
eeg = None
# f.create_dataset('label', data=[1], compression="gzip", compression_opts=compresslevel)
# f.create_dataset('pred_window', data=[event[2]], compression="gzip", compression_opts=compresslevel)
# f.create_dataset('caseid', data=[caseid], compression="gzip", compression_opts=compresslevel)
elif debug:
print(f"{caseid:04d} {predWindow:02d}min {startIndex} starttime = ignored, segment validity issues")
count_neg_saved = 0
for i in range(0, len(negativeSegments)):
event = negativeSegments[i]
startIndex = event[0]
endIndex = event[1]
predWindow = event[2]
abp = event[3]
#ecg = event[4]
#eeg = event[5]
seg_filename = f"{caseid:04d}_{startIndex}_0_False.h5"
seg_fullpath = f"{seg_folder}/{seg_filename}"
if isAbpSegmentValidNumpy(abp, debug):
count_neg_saved += 1
abp = abp.tolist()
ecg = event[4].tolist()
eeg = event[5].tolist()
f = h5py.File(seg_fullpath, "w")
f.create_dataset('abp', data=abp, compression="gzip", compression_opts=compresslevel)
f.create_dataset('ecg', data=ecg, compression="gzip", compression_opts=compresslevel)
f.create_dataset('eeg', data=eeg, compression="gzip", compression_opts=compresslevel)
f.flush()
f.close()
f = None
abp = None
ecg = None
eeg = None
# f.create_dataset('label', data=[0], compression="gzip", compression_opts=compresslevel)
# f.create_dataset('pred_window', data=[0], compression="gzip", compression_opts=compresslevel)
# f.create_dataset('caseid', data=[caseid], compression="gzip", compression_opts=compresslevel)
elif debug:
print(f"{caseid:04d} CleanWindow {startIndex} starttime = ignored, segment validity issues")
if count_neg_saved == 0 and count_pos_saved == 0:
print(f'{caseid}: nothing saved, all segments filtered')
# Generate hypotensive events
# Hypotensive events are defined as a 1-minute interval with sustained ABP of less than 65 mmHg
# Note: Hypotensive events should be at least 20 minutes apart to minimize potential residual effects from previous events
# Generate hypotension non-events
# To sample non-events, 30-minute segments where the ABP was above 75 mmHG were selected, and then
# three one-minute samples of each waveform were obtained from the middle of the segment
# both occur in extract_segments
#VITAL_EXTRACTED_SEGMENTS
def extract_segments(
cases_of_interest_idx,
debug=False,
checkCache=True,
forceWrite=False,
returnSegments=False,
skipInvalidCleanEvents=False
):
# Sampling rate for ABP and ECG, Hz. These rates should be the same. Default = 500
ABP_ECG_SRATE_HZ = 500
# Sampling rate for EEG. Default = 128
EEG_SRATE_HZ = 128
# Final dataset for training and testing the model.
positiveSegmentsMap = {}
negativeSegmentsMap = {}
iohEventsMap = {}
cleanEventsMap = {}
# Process each case and extract segments. For each segment identify presence of an event in the label zone.
count_cases = len(cases_of_interest_idx)
#for case_count, caseid in tqdm(enumerate(cases_of_interest_idx), total=count_cases):
for case_count, caseid in enumerate(cases_of_interest_idx):
if debug:
print(f'Loading case: {caseid:04d}, ({case_count + 1} of {count_cases})')
if checkCache and areCaseSegmentsCached(caseid):
if debug:
print(f'Skipping case: {caseid:04d}, already cached')
# skip records we've already cached
continue
# read the arterial waveform
(abp, ecg, eeg, event) = get_track_data(caseid)
if debug:
print(f'Length of {TRACK_NAMES[0]}: {abp.shape[0]}')
print(f'Length of {TRACK_NAMES[1]}: {ecg.shape[0]}')
print(f'Length of {TRACK_NAMES[2]}: {eeg.shape[0]}')
(startInSeconds, endInSeconds) = getSurgeryBoundariesInSeconds(event)
if debug:
print(f"Event markers indicate that surgery begins at {startInSeconds}s and ends at {endInSeconds}s.")
track_length_seconds = int(len(abp) / ABP_ECG_SRATE_HZ)
if debug:
print(f"Processing case {caseid} with length {track_length_seconds}s")
# check if the ABP segment in the surgery window is valid
if debug:
isSurgerySegmentValid = isAbpSegmentValidNumpy(abp[startInSeconds:endInSeconds])
print(f'{caseid}: surgery segment valid: {isSurgerySegmentValid}')
iohEvents = []
cleanEvents = []
i = 0
started = False
eofReached = False
trackStartIndex = None
# set i pointer (which operates in seconds) to start marker for surgery
i = startInSeconds
# FIRST PASS
# in the first forward pass, we are going to identify the start/end boundaries of all IOH events within the case
while i < track_length_seconds - 60 and i < endInSeconds:
segmentStart = None
segmentEnd = None
segFound = False
# look forward one minute
abpSeg = abp[i * ABP_ECG_SRATE_HZ:(i + 60) * ABP_ECG_SRATE_HZ]
# roll forward until we hit a one minute window where mean ABP >= 65 so we know leads are connected and it's tracking
if not started:
if np.nanmean(abpSeg) >= 65:
started = True
trackStartIndex = i
# if we're started and mean abp for the window is <65, we are starting a new IOH event
elif np.nanmean(abpSeg) < 65:
segmentStart = i
# now seek forward to find end of event, perpetually checking the lats minute of the IOH event
for j in range(i + 60, track_length_seconds):
# look backward one minute
abpSegForward = abp[(j - 60) * ABP_ECG_SRATE_HZ:j * ABP_ECG_SRATE_HZ]
if np.nanmean(abpSegForward) >= 65:
segmentEnd = j - 1
break
if segmentEnd is None:
eofReached = True
else:
# otherwise, end of the IOH segment has been reached, record it
iohEvents.append((segmentStart, segmentEnd))
segFound = True
if debug:
t_abp = abp[segmentStart * ABP_ECG_SRATE_HZ:segmentEnd * ABP_ECG_SRATE_HZ]
isIohSegmentValid = isAbpSegmentValidNumpy(t_abp)
print(f'{caseid}: ioh segment valid: {isIohSegmentValid}, {segmentStart}, {segmentEnd}, {t_abp.shape}')
i += 1
if not started:
continue
elif eofReached:
break
elif segFound:
i = segmentEnd + 1
# SECOND PASS
# in the second forward pass, we are going to identify the start/end boundaries of all non-overlapping 30 minute "clean" windows
# reuse the 'start of signal' index from our first pass
if trackStartIndex is None:
trackStartIndex = startInSeconds
i = trackStartIndex
eofReached = False
clean_events_valid = []
while i < track_length_seconds - 1800 and i < endInSeconds:
segmentStart = None
segmentEnd = None
segFound = False
startIndex = i
endIndex = i + 1800
# check to see if this 30 minute window overlaps any IOH events, if so ffwd to end of latest overlapping IOH
overlapFound = False
latestEnd = None
for event in iohEvents:
# case 1: starts during an event
if startIndex >= event[0] and startIndex < event[1]:
latestEnd = event[1]
overlapFound = True
# case 2: ends during an event
elif endIndex >= event[0] and endIndex < event[1]:
latestEnd = event[1]
overlapFound = True
# case 3: event occurs entirely inside of the window
elif startIndex < event[0] and endIndex > event[1]:
latestEnd = event[1]
overlapFound = True
# FFWD if we found an overlap
if overlapFound:
i = latestEnd + 1
continue
# look forward 30 minutes
abpSeg = abp[startIndex * ABP_ECG_SRATE_HZ:endIndex * ABP_ECG_SRATE_HZ]
# if we're started and mean abp for the window is >= 75, we are starting a new clean event
if np.nanmean(abpSeg) >= 75:
overlapFound = False
latestEnd = None
for event in iohEvents:
# case 1: starts during an event
if startIndex >= event[0] and startIndex < event[1]:
latestEnd = event[1]
overlapFound = True
# case 2: ends during an event
elif endIndex >= event[0] and endIndex < event[1]:
latestEnd = event[1]
overlapFound = True
# case 3: event occurs entirely inside of the window
elif startIndex < event[0] and endIndex > event[1]:
latestEnd = event[1]
overlapFound = True
if not overlapFound:
segFound = True
segmentEnd = endIndex
cleanEvents.append((startIndex, endIndex))
if skipInvalidCleanEvents:
isCleanSegmentValid = isAbpSegmentValidNumpy(abpSeg)
clean_events_valid.append(isCleanSegmentValid)
if debug:
print(f'{caseid}: clean segment valid: {isCleanSegmentValid}, {startIndex}, {endIndex}, {abpSeg.shape}')
else:
clean_events_valid.append(True)
i += 10
if segFound:
i = segmentEnd + 1
if debug:
print(f"IOH Events for case {caseid}: {iohEvents}")
print(f"Clean Events for case {caseid}: {cleanEvents}")
positiveSegments = []
negativeSegments = []
# THIRD PASS
# in the third pass, we will use the collections of ioh event windows to generate our actual extracted segments based on our prediction window (positive labels)
for i in range(0, len(iohEvents)):
if debug:
print(f"Checking event {iohEvents[i]}")
# we want to review current event boundaries, as well as previous event boundaries if available
event = iohEvents[i]
previousEvent = None
if i > 0:
previousEvent = iohEvents[i - 1]
for predWindow in ALL_PREDICTION_WINDOWS:
if debug:
print(f"Checking event {iohEvents[i]} for pred {predWindow}")
iohEventStart = event[0]
predictiveSegmentEnd = event[0] - (predWindow*60)
predictiveSegmentStart = predictiveSegmentEnd - 60
if (predictiveSegmentStart < 0):
# don't rewind before the beginning of the track
if debug:
print(f"Checking event {iohEvents[i]} for pred {predWindow} - exit, before beginning")
continue
elif (predictiveSegmentStart < trackStartIndex):
# don't rewind before the beginning of signal in track
if debug:
print(f"Checking event {iohEvents[i]} for pred {predWindow} - exit, before track start")
continue
elif previousEvent is not None:
# does this event window come before or during the previous event?
overlapFound = False
# case 1: starts during an event
if predictiveSegmentStart >= previousEvent[0] and predictiveSegmentStart < previousEvent[1]:
overlapFound = True
# case 2: ends during an event
elif iohEventStart >= previousEvent[0] and iohEventStart < previousEvent[1]:
overlapFound = True
# case 3: event occurs entirely inside of the window
elif predictiveSegmentStart < previousEvent[0] and iohEventStart > previousEvent[1]:
overlapFound = True
# do not extract a case if we overlap witha nother IOH
if overlapFound:
if debug:
print(f"Checking event {iohEvents[i]} for pred {predWindow} - exit, overlap with earlier segment")
continue
# track the positive segment
positiveSegments.append((predictiveSegmentStart, predictiveSegmentEnd, predWindow,
abp[predictiveSegmentStart*ABP_ECG_SRATE_HZ:predictiveSegmentEnd*ABP_ECG_SRATE_HZ],
ecg[predictiveSegmentStart*ABP_ECG_SRATE_HZ:predictiveSegmentEnd*ABP_ECG_SRATE_HZ],
eeg[predictiveSegmentStart*EEG_SRATE_HZ:predictiveSegmentEnd*EEG_SRATE_HZ]))
# FOURTH PASS
# in the fourth and final pass, we will use the collections of clean event windows to generate our actual extracted segments based (negative labels)
for i in range(0, len(cleanEvents)):
# Don't extract segments from invalid clean event windows.
if not clean_events_valid[i]:
continue
# everything will be 30 minutes long at least
event = cleanEvents[i]
# choose sample 1 @ 10 minutes
# choose sample 2 @ 15 minutes
# choose sample 3 @ 20 minutes
timeAtTen = event[0] + 600
timeAtFifteen = event[0] + 900
timeAtTwenty = event[0] + 1200
negativeSegments.append((timeAtTen, timeAtTen + 60, 0,
abp[timeAtTen*ABP_ECG_SRATE_HZ:(timeAtTen + 60)*ABP_ECG_SRATE_HZ],
ecg[timeAtTen*ABP_ECG_SRATE_HZ:(timeAtTen + 60)*ABP_ECG_SRATE_HZ],
eeg[timeAtTen*EEG_SRATE_HZ:(timeAtTen + 60)*EEG_SRATE_HZ]))
negativeSegments.append((timeAtFifteen, timeAtFifteen + 60, 0,
abp[timeAtFifteen*ABP_ECG_SRATE_HZ:(timeAtFifteen + 60)*ABP_ECG_SRATE_HZ],
ecg[timeAtFifteen*ABP_ECG_SRATE_HZ:(timeAtFifteen + 60)*ABP_ECG_SRATE_HZ],
eeg[timeAtFifteen*EEG_SRATE_HZ:(timeAtFifteen + 60)*EEG_SRATE_HZ]))
negativeSegments.append((timeAtTwenty, timeAtTwenty + 60, 0,
abp[timeAtTwenty*ABP_ECG_SRATE_HZ:(timeAtTwenty + 60)*ABP_ECG_SRATE_HZ],
ecg[timeAtTwenty*ABP_ECG_SRATE_HZ:(timeAtTwenty + 60)*ABP_ECG_SRATE_HZ],
eeg[timeAtTwenty*EEG_SRATE_HZ:(timeAtTwenty + 60)*EEG_SRATE_HZ]))
if returnSegments:
positiveSegmentsMap[caseid] = positiveSegments
negativeSegmentsMap[caseid] = negativeSegments
iohEventsMap[caseid] = iohEvents
cleanEventsMap[caseid] = cleanEvents
saveCaseSegments(caseid, positiveSegments, negativeSegments, 9, debug=debug, forceWrite=forceWrite)
#if debug:
print(f'{caseid}: positiveSegments: {len(positiveSegments)}, negativeSegments: {len(negativeSegments)}')
return positiveSegmentsMap, negativeSegmentsMap, iohEventsMap, cleanEventsMap
Ensure that all needed segments are in place for the cases that are being used. If data is already stored on disk this method returns immediately.
print('Time to extract segments!')
Time to extract segments!
MANUAL_EXTRACT=True
SKIP_INVALID_CLEAN_EVENTS=True
if MANUAL_EXTRACT:
mycoi = cases_of_interest_idx
#mycoi = cases_of_interest_idx[:2800]
#mycoi = [1]
cnt = 0
mod = 0
for ci in mycoi:
cnt += 1
if mod % 100 == 0:
print(f'count processed: {mod}, current case index: {ci}')
try:
p, n, i, c = extract_segments([ci], debug=False, checkCache=True,
forceWrite=True, returnSegments=False,
skipInvalidCleanEvents=SKIP_INVALID_CLEAN_EVENTS)
p = None
n = None
i = None
c = None
except:
print(f'error on extract segment: {ci}')
mod += 1
print(f'extracted: {cnt}')
count processed: 0, current case index: 1 1: positiveSegments: 12, negativeSegments: 3 4: positiveSegments: 22, negativeSegments: 3 7: positiveSegments: 12, negativeSegments: 6 10: positiveSegments: 27, negativeSegments: 6 12: positiveSegments: 22, negativeSegments: 0 13: positiveSegments: 18, negativeSegments: 0 16: positiveSegments: 12, negativeSegments: 6 19: positiveSegments: 34, negativeSegments: 3 20: positiveSegments: 17, negativeSegments: 9 22: positiveSegments: 16, negativeSegments: 12 24: positiveSegments: 3, negativeSegments: 6 25: positiveSegments: 6, negativeSegments: 12 26: exit early, no segments to save 26: positiveSegments: 0, negativeSegments: 0 27: positiveSegments: 11, negativeSegments: 12 29: positiveSegments: 8, negativeSegments: 12 31: positiveSegments: 4, negativeSegments: 3 34: positiveSegments: 4, negativeSegments: 9 38: positiveSegments: 10, negativeSegments: 0 43: positiveSegments: 14, negativeSegments: 3 44: positiveSegments: 4, negativeSegments: 6 46: positiveSegments: 0, negativeSegments: 3 49: positiveSegments: 4, negativeSegments: 0 50: positiveSegments: 12, negativeSegments: 6 52: positiveSegments: 20, negativeSegments: 3 53: positiveSegments: 0, negativeSegments: 9 55: positiveSegments: 22, negativeSegments: 6 58: positiveSegments: 14, negativeSegments: 0 59: positiveSegments: 3, negativeSegments: 0 60: positiveSegments: 7, negativeSegments: 3 61: positiveSegments: 10, negativeSegments: 3 64: positiveSegments: 16, negativeSegments: 9 65: positiveSegments: 3, negativeSegments: 3 66: positiveSegments: 14, negativeSegments: 6 67: positiveSegments: 9, negativeSegments: 0 68: positiveSegments: 0, negativeSegments: 3 69: exit early, no segments to save 69: positiveSegments: 0, negativeSegments: 0 70: exit early, no segments to save 70: positiveSegments: 0, negativeSegments: 0 74: positiveSegments: 4, negativeSegments: 6 75: positiveSegments: 42, negativeSegments: 3 77: positiveSegments: 0, negativeSegments: 9 79: positiveSegments: 14, negativeSegments: 12 83: positiveSegments: 13, negativeSegments: 0 84: positiveSegments: 4, negativeSegments: 15 87: positiveSegments: 9, negativeSegments: 0 89: positiveSegments: 5, negativeSegments: 18 92: positiveSegments: 1, negativeSegments: 0 93: positiveSegments: 7, negativeSegments: 0 94: positiveSegments: 30, negativeSegments: 6 96: positiveSegments: 33, negativeSegments: 18 97: positiveSegments: 14, negativeSegments: 0 98: positiveSegments: 8, negativeSegments: 3 101: positiveSegments: 0, negativeSegments: 9 104: positiveSegments: 15, negativeSegments: 0 105: positiveSegments: 23, negativeSegments: 0 108: positiveSegments: 5, negativeSegments: 0 110: positiveSegments: 11, negativeSegments: 0 111: positiveSegments: 11, negativeSegments: 0 112: positiveSegments: 13, negativeSegments: 0 114: positiveSegments: 8, negativeSegments: 9 116: positiveSegments: 15, negativeSegments: 0 117: positiveSegments: 15, negativeSegments: 3 118: positiveSegments: 48, negativeSegments: 0 119: positiveSegments: 2, negativeSegments: 9 124: positiveSegments: 2, negativeSegments: 3 125: positiveSegments: 4, negativeSegments: 9 126: exit early, no segments to save 126: positiveSegments: 0, negativeSegments: 0 128: positiveSegments: 7, negativeSegments: 3 130: positiveSegments: 0, negativeSegments: 9 132: positiveSegments: 0, negativeSegments: 3 135: positiveSegments: 13, negativeSegments: 0 136: positiveSegments: 10, negativeSegments: 9 137: positiveSegments: 4, negativeSegments: 0 138: positiveSegments: 0, negativeSegments: 9 139: exit early, no segments to save 139: positiveSegments: 0, negativeSegments: 0 140: positiveSegments: 12, negativeSegments: 12 142: positiveSegments: 5, negativeSegments: 9 143: positiveSegments: 12, negativeSegments: 3 145: positiveSegments: 0, negativeSegments: 6 146: positiveSegments: 14, negativeSegments: 0 148: positiveSegments: 15, negativeSegments: 0 149: positiveSegments: 8, negativeSegments: 0 152: positiveSegments: 5, negativeSegments: 0 153: positiveSegments: 12, negativeSegments: 0 156: positiveSegments: 9, negativeSegments: 12 161: positiveSegments: 12, negativeSegments: 3 163: positiveSegments: 5, negativeSegments: 0 166: positiveSegments: 18, negativeSegments: 3 167: positiveSegments: 0, negativeSegments: 3 172: exit early, no segments to save 172: positiveSegments: 0, negativeSegments: 0 175: positiveSegments: 3, negativeSegments: 3 177: positiveSegments: 12, negativeSegments: 9 178: positiveSegments: 1, negativeSegments: 12 181: positiveSegments: 8, negativeSegments: 3 183: positiveSegments: 4, negativeSegments: 0 184: positiveSegments: 46, negativeSegments: 12 186: positiveSegments: 0, negativeSegments: 3 190: positiveSegments: 4, negativeSegments: 3 191: positiveSegments: 16, negativeSegments: 0 195: positiveSegments: 21, negativeSegments: 3 197: positiveSegments: 14, negativeSegments: 6 count processed: 100, current case index: 198 198: positiveSegments: 8, negativeSegments: 12 199: exit early, no segments to save 199: positiveSegments: 0, negativeSegments: 0 200: positiveSegments: 0, negativeSegments: 6 202: positiveSegments: 0, negativeSegments: 15 203: positiveSegments: 6, negativeSegments: 0 206: positiveSegments: 8, negativeSegments: 6 208: positiveSegments: 8, negativeSegments: 0 210: positiveSegments: 4, negativeSegments: 6 218: exit early, no segments to save 218: positiveSegments: 0, negativeSegments: 0 221: exit early, no segments to save 221: positiveSegments: 0, negativeSegments: 0 222: positiveSegments: 2, negativeSegments: 3 229: positiveSegments: 0, negativeSegments: 15 232: positiveSegments: 4, negativeSegments: 3 233: positiveSegments: 14, negativeSegments: 0 234: positiveSegments: 4, negativeSegments: 9 236: positiveSegments: 13, negativeSegments: 12 237: positiveSegments: 2, negativeSegments: 9 239: positiveSegments: 0, negativeSegments: 6 241: positiveSegments: 31, negativeSegments: 12 244: positiveSegments: 14, negativeSegments: 0 247: positiveSegments: 0, negativeSegments: 18 250: positiveSegments: 0, negativeSegments: 9 251: positiveSegments: 18, negativeSegments: 0 252: positiveSegments: 17, negativeSegments: 9 256: positiveSegments: 16, negativeSegments: 3 258: exit early, no segments to save 258: positiveSegments: 0, negativeSegments: 0 261: positiveSegments: 4, negativeSegments: 6 263: positiveSegments: 0, negativeSegments: 3 266: positiveSegments: 3, negativeSegments: 9 268: positiveSegments: 2, negativeSegments: 3 269: positiveSegments: 8, negativeSegments: 6 270: positiveSegments: 2, negativeSegments: 0 279: positiveSegments: 3, negativeSegments: 6 281: positiveSegments: 5, negativeSegments: 6 282: positiveSegments: 0, negativeSegments: 3 283: positiveSegments: 14, negativeSegments: 3 286: positiveSegments: 0, negativeSegments: 9 287: positiveSegments: 4, negativeSegments: 6 293: positiveSegments: 2, negativeSegments: 3 295: positiveSegments: 13, negativeSegments: 3 296: positiveSegments: 4, negativeSegments: 6 297: positiveSegments: 0, negativeSegments: 3 300: positiveSegments: 9, negativeSegments: 3 303: positiveSegments: 11, negativeSegments: 9 304: positiveSegments: 6, negativeSegments: 9 306: positiveSegments: 8, negativeSegments: 12 308: positiveSegments: 4, negativeSegments: 6 309: positiveSegments: 13, negativeSegments: 0 312: positiveSegments: 4, negativeSegments: 6 316: positiveSegments: 7, negativeSegments: 0 318: positiveSegments: 9, negativeSegments: 3 319: exit early, no segments to save 319: positiveSegments: 0, negativeSegments: 0 321: positiveSegments: 6, negativeSegments: 9 323: positiveSegments: 12, negativeSegments: 3 327: positiveSegments: 43, negativeSegments: 0 330: positiveSegments: 4, negativeSegments: 6 337: positiveSegments: 8, negativeSegments: 3 338: exit early, no segments to save 338: positiveSegments: 0, negativeSegments: 0 342: positiveSegments: 2, negativeSegments: 6 343: positiveSegments: 3, negativeSegments: 6 345: positiveSegments: 0, negativeSegments: 15 347: exit early, no segments to save 347: positiveSegments: 0, negativeSegments: 0 348: positiveSegments: 12, negativeSegments: 3 349: positiveSegments: 8, negativeSegments: 0 353: positiveSegments: 0, negativeSegments: 12 354: positiveSegments: 22, negativeSegments: 0 355: positiveSegments: 9, negativeSegments: 3 357: positiveSegments: 4, negativeSegments: 3 358: positiveSegments: 16, negativeSegments: 0 359: positiveSegments: 37, negativeSegments: 6 362: positiveSegments: 4, negativeSegments: 12 363: positiveSegments: 16, negativeSegments: 3 367: positiveSegments: 4, negativeSegments: 3 369: positiveSegments: 4, negativeSegments: 9 370: positiveSegments: 4, negativeSegments: 0 371: positiveSegments: 4, negativeSegments: 3 375: positiveSegments: 31, negativeSegments: 0 380: positiveSegments: 4, negativeSegments: 0 382: positiveSegments: 6, negativeSegments: 15 383: positiveSegments: 12, negativeSegments: 0 384: positiveSegments: 14, negativeSegments: 0 387: positiveSegments: 3, negativeSegments: 0 388: positiveSegments: 4, negativeSegments: 6 390: positiveSegments: 13, negativeSegments: 9 397: positiveSegments: 20, negativeSegments: 0 398: positiveSegments: 0, negativeSegments: 3 402: positiveSegments: 4, negativeSegments: 6 404: positiveSegments: 0, negativeSegments: 3 405: positiveSegments: 2, negativeSegments: 0 406: positiveSegments: 4, negativeSegments: 6 408: positiveSegments: 4, negativeSegments: 0 409: positiveSegments: 16, negativeSegments: 0 413: exit early, no segments to save 413: positiveSegments: 0, negativeSegments: 0 415: positiveSegments: 9, negativeSegments: 0 416: positiveSegments: 14, negativeSegments: 0 417: positiveSegments: 9, negativeSegments: 15 418: positiveSegments: 28, negativeSegments: 0 419: positiveSegments: 3, negativeSegments: 6 425: exit early, no segments to save 425: positiveSegments: 0, negativeSegments: 0 427: positiveSegments: 0, negativeSegments: 3 count processed: 200, current case index: 431 431: positiveSegments: 9, negativeSegments: 0 435: positiveSegments: 7, negativeSegments: 6 439: positiveSegments: 7, negativeSegments: 0 440: positiveSegments: 4, negativeSegments: 6 441: positiveSegments: 4, negativeSegments: 0 442: positiveSegments: 6, negativeSegments: 0 445: positiveSegments: 12, negativeSegments: 18 447: positiveSegments: 2, negativeSegments: 3 448: positiveSegments: 4, negativeSegments: 12 449: positiveSegments: 16, negativeSegments: 0 451: positiveSegments: 21, negativeSegments: 3 452: positiveSegments: 9, negativeSegments: 3 455: positiveSegments: 4, negativeSegments: 0 458: positiveSegments: 4, negativeSegments: 3 462: positiveSegments: 9, negativeSegments: 0 466: positiveSegments: 6, negativeSegments: 12 469: positiveSegments: 34, negativeSegments: 3 472: positiveSegments: 10, negativeSegments: 21 474: positiveSegments: 28, negativeSegments: 9 476: positiveSegments: 26, negativeSegments: 3 478: positiveSegments: 5, negativeSegments: 3 481: positiveSegments: 10, negativeSegments: 3 484: positiveSegments: 0, negativeSegments: 9 485: positiveSegments: 6, negativeSegments: 0 486: positiveSegments: 14, negativeSegments: 3 488: positiveSegments: 14, negativeSegments: 0 490: positiveSegments: 7, negativeSegments: 6 492: positiveSegments: 16, negativeSegments: 12 495: positiveSegments: 4, negativeSegments: 0 499: positiveSegments: 32, negativeSegments: 6 505: positiveSegments: 23, negativeSegments: 3 512: positiveSegments: 5, negativeSegments: 18 513: positiveSegments: 13, negativeSegments: 3 516: positiveSegments: 0, negativeSegments: 6 520: positiveSegments: 10, negativeSegments: 9 521: positiveSegments: 12, negativeSegments: 0 526: positiveSegments: 7, negativeSegments: 6 527: positiveSegments: 20, negativeSegments: 0 530: positiveSegments: 7, negativeSegments: 3 531: positiveSegments: 3, negativeSegments: 0 535: positiveSegments: 0, negativeSegments: 3 536: positiveSegments: 0, negativeSegments: 3 537: positiveSegments: 4, negativeSegments: 0 541: exit early, no segments to save 541: positiveSegments: 0, negativeSegments: 0 543: positiveSegments: 13, negativeSegments: 3 544: positiveSegments: 4, negativeSegments: 6 545: positiveSegments: 2, negativeSegments: 6 547: exit early, no segments to save 547: positiveSegments: 0, negativeSegments: 0 550: positiveSegments: 11, negativeSegments: 6 551: positiveSegments: 18, negativeSegments: 3 553: positiveSegments: 30, negativeSegments: 3 559: positiveSegments: 12, negativeSegments: 6 560: positiveSegments: 0, negativeSegments: 6 561: positiveSegments: 4, negativeSegments: 0 562: positiveSegments: 6, negativeSegments: 6 563: positiveSegments: 4, negativeSegments: 0 564: positiveSegments: 8, negativeSegments: 6 566: positiveSegments: 11, negativeSegments: 6 567: positiveSegments: 0, negativeSegments: 9 568: positiveSegments: 42, negativeSegments: 0 570: positiveSegments: 8, negativeSegments: 9 573: positiveSegments: 8, negativeSegments: 6 576: positiveSegments: 4, negativeSegments: 3 577: positiveSegments: 5, negativeSegments: 9 579: positiveSegments: 4, negativeSegments: 0 582: positiveSegments: 0, negativeSegments: 3 584: nothing saved, all segments filtered 584: positiveSegments: 11, negativeSegments: 0 585: positiveSegments: 5, negativeSegments: 3 587: positiveSegments: 20, negativeSegments: 3 590: positiveSegments: 6, negativeSegments: 3 593: positiveSegments: 2, negativeSegments: 6 594: exit early, no segments to save 594: positiveSegments: 0, negativeSegments: 0 599: positiveSegments: 8, negativeSegments: 3 611: positiveSegments: 13, negativeSegments: 9 612: positiveSegments: 4, negativeSegments: 9 616: positiveSegments: 7, negativeSegments: 0 617: positiveSegments: 4, negativeSegments: 3 620: positiveSegments: 0, negativeSegments: 12 621: positiveSegments: 7, negativeSegments: 0 622: positiveSegments: 0, negativeSegments: 6 624: positiveSegments: 11, negativeSegments: 0 627: positiveSegments: 10, negativeSegments: 6 628: positiveSegments: 36, negativeSegments: 0 629: positiveSegments: 40, negativeSegments: 6 631: positiveSegments: 0, negativeSegments: 15 634: positiveSegments: 4, negativeSegments: 6 636: positiveSegments: 0, negativeSegments: 3 637: positiveSegments: 0, negativeSegments: 15 641: exit early, no segments to save 641: positiveSegments: 0, negativeSegments: 0 644: positiveSegments: 6, negativeSegments: 3 645: positiveSegments: 4, negativeSegments: 3 648: positiveSegments: 8, negativeSegments: 0 649: positiveSegments: 21, negativeSegments: 3 650: positiveSegments: 4, negativeSegments: 0 652: positiveSegments: 17, negativeSegments: 3 655: positiveSegments: 0, negativeSegments: 6 657: positiveSegments: 6, negativeSegments: 3 659: positiveSegments: 4, negativeSegments: 6 660: positiveSegments: 0, negativeSegments: 6 663: positiveSegments: 0, negativeSegments: 6 count processed: 300, current case index: 665 665: exit early, no segments to save 665: positiveSegments: 0, negativeSegments: 0 666: positiveSegments: 11, negativeSegments: 12 667: positiveSegments: 4, negativeSegments: 9 671: positiveSegments: 6, negativeSegments: 15 672: positiveSegments: 0, negativeSegments: 3 676: positiveSegments: 8, negativeSegments: 6 679: positiveSegments: 2, negativeSegments: 0 680: positiveSegments: 2, negativeSegments: 0 683: positiveSegments: 0, negativeSegments: 6 684: positiveSegments: 4, negativeSegments: 3 685: positiveSegments: 3, negativeSegments: 9 687: positiveSegments: 4, negativeSegments: 0 689: exit early, no segments to save 689: positiveSegments: 0, negativeSegments: 0 691: positiveSegments: 5, negativeSegments: 12 697: positiveSegments: 16, negativeSegments: 0 698: positiveSegments: 21, negativeSegments: 3 699: positiveSegments: 21, negativeSegments: 3 702: positiveSegments: 21, negativeSegments: 12 703: positiveSegments: 18, negativeSegments: 3 706: positiveSegments: 18, negativeSegments: 0 711: positiveSegments: 0, negativeSegments: 18 716: positiveSegments: 8, negativeSegments: 3 719: positiveSegments: 4, negativeSegments: 15 721: positiveSegments: 18, negativeSegments: 0 722: positiveSegments: 4, negativeSegments: 6 724: exit early, no segments to save 724: positiveSegments: 0, negativeSegments: 0 725: positiveSegments: 14, negativeSegments: 15 726: positiveSegments: 0, negativeSegments: 3 728: positiveSegments: 25, negativeSegments: 15 729: positiveSegments: 4, negativeSegments: 3 730: positiveSegments: 9, negativeSegments: 3 733: positiveSegments: 12, negativeSegments: 3 734: positiveSegments: 1, negativeSegments: 15 737: positiveSegments: 4, negativeSegments: 0 739: positiveSegments: 0, negativeSegments: 3 740: positiveSegments: 6, negativeSegments: 3 742: positiveSegments: 4, negativeSegments: 6 744: positiveSegments: 0, negativeSegments: 3 745: exit early, no segments to save 745: positiveSegments: 0, negativeSegments: 0 746: positiveSegments: 7, negativeSegments: 3 747: exit early, no segments to save 747: positiveSegments: 0, negativeSegments: 0 748: positiveSegments: 20, negativeSegments: 12 749: positiveSegments: 2, negativeSegments: 0 750: positiveSegments: 59, negativeSegments: 12 751: positiveSegments: 5, negativeSegments: 15 752: positiveSegments: 2, negativeSegments: 12 753: positiveSegments: 4, negativeSegments: 0 755: positiveSegments: 8, negativeSegments: 3 756: positiveSegments: 4, negativeSegments: 9 757: positiveSegments: 2, negativeSegments: 6 758: positiveSegments: 11, negativeSegments: 0 761: positiveSegments: 4, negativeSegments: 0 762: positiveSegments: 7, negativeSegments: 9 763: positiveSegments: 21, negativeSegments: 6 764: positiveSegments: 26, negativeSegments: 6 765: positiveSegments: 25, negativeSegments: 9 767: positiveSegments: 4, negativeSegments: 6 768: positiveSegments: 7, negativeSegments: 0 770: positiveSegments: 5, negativeSegments: 0 772: positiveSegments: 0, negativeSegments: 3 773: positiveSegments: 0, negativeSegments: 3 774: positiveSegments: 22, negativeSegments: 3 775: positiveSegments: 22, negativeSegments: 0 776: positiveSegments: 12, negativeSegments: 3 777: positiveSegments: 23, negativeSegments: 3 779: positiveSegments: 5, negativeSegments: 12 781: positiveSegments: 7, negativeSegments: 0 783: positiveSegments: 4, negativeSegments: 0 788: positiveSegments: 4, negativeSegments: 3 792: positiveSegments: 6, negativeSegments: 0 793: positiveSegments: 21, negativeSegments: 0 794: positiveSegments: 13, negativeSegments: 0 795: positiveSegments: 9, negativeSegments: 9 797: exit early, no segments to save 797: positiveSegments: 0, negativeSegments: 0 800: positiveSegments: 15, negativeSegments: 3 802: positiveSegments: 24, negativeSegments: 6 807: positiveSegments: 8, negativeSegments: 6 808: positiveSegments: 14, negativeSegments: 3 810: positiveSegments: 2, negativeSegments: 0 812: positiveSegments: 4, negativeSegments: 3 813: positiveSegments: 9, negativeSegments: 3 814: positiveSegments: 7, negativeSegments: 12 815: positiveSegments: 0, negativeSegments: 18 816: positiveSegments: 4, negativeSegments: 12 818: exit early, no segments to save 818: positiveSegments: 0, negativeSegments: 0 819: positiveSegments: 14, negativeSegments: 0 822: positiveSegments: 18, negativeSegments: 15 825: positiveSegments: 0, negativeSegments: 9 827: positiveSegments: 4, negativeSegments: 0 830: positiveSegments: 2, negativeSegments: 6 831: positiveSegments: 0, negativeSegments: 6 833: positiveSegments: 13, negativeSegments: 3 835: positiveSegments: 0, negativeSegments: 3 841: positiveSegments: 0, negativeSegments: 9 843: positiveSegments: 28, negativeSegments: 9 846: positiveSegments: 14, negativeSegments: 0 847: positiveSegments: 0, negativeSegments: 12 848: positiveSegments: 4, negativeSegments: 0 851: positiveSegments: 25, negativeSegments: 3 852: positiveSegments: 1, negativeSegments: 3 count processed: 400, current case index: 853 853: positiveSegments: 4, negativeSegments: 6 855: positiveSegments: 8, negativeSegments: 3 859: positiveSegments: 8, negativeSegments: 9 860: positiveSegments: 12, negativeSegments: 3 864: positiveSegments: 11, negativeSegments: 0 865: positiveSegments: 11, negativeSegments: 6 866: positiveSegments: 1, negativeSegments: 6 868: positiveSegments: 10, negativeSegments: 0 869: positiveSegments: 16, negativeSegments: 12 870: positiveSegments: 19, negativeSegments: 3 871: positiveSegments: 2, negativeSegments: 6 872: positiveSegments: 0, negativeSegments: 6 876: positiveSegments: 2, negativeSegments: 9 879: positiveSegments: 5, negativeSegments: 3 880: positiveSegments: 8, negativeSegments: 3 881: exit early, no segments to save 881: positiveSegments: 0, negativeSegments: 0 883: positiveSegments: 21, negativeSegments: 6 885: positiveSegments: 4, negativeSegments: 15 886: positiveSegments: 14, negativeSegments: 18 887: positiveSegments: 4, negativeSegments: 3 890: positiveSegments: 6, negativeSegments: 0 892: positiveSegments: 10, negativeSegments: 3 894: positiveSegments: 4, negativeSegments: 0 898: positiveSegments: 0, negativeSegments: 3 907: positiveSegments: 13, negativeSegments: 9 912: positiveSegments: 9, negativeSegments: 3 913: positiveSegments: 5, negativeSegments: 3 916: positiveSegments: 19, negativeSegments: 6 917: positiveSegments: 4, negativeSegments: 3 919: positiveSegments: 0, negativeSegments: 6 922: positiveSegments: 3, negativeSegments: 3 925: exit early, no segments to save 925: positiveSegments: 0, negativeSegments: 0 926: positiveSegments: 5, negativeSegments: 0 931: positiveSegments: 5, negativeSegments: 0 932: positiveSegments: 10, negativeSegments: 3 933: exit early, no segments to save 933: positiveSegments: 0, negativeSegments: 0 936: positiveSegments: 9, negativeSegments: 3 937: positiveSegments: 5, negativeSegments: 12 938: positiveSegments: 8, negativeSegments: 0 939: positiveSegments: 12, negativeSegments: 3 940: positiveSegments: 8, negativeSegments: 0 944: positiveSegments: 16, negativeSegments: 12 945: positiveSegments: 26, negativeSegments: 3 946: positiveSegments: 12, negativeSegments: 0 947: positiveSegments: 6, negativeSegments: 15 948: positiveSegments: 11, negativeSegments: 6 949: positiveSegments: 10, negativeSegments: 3 952: exit early, no segments to save 952: positiveSegments: 0, negativeSegments: 0 954: positiveSegments: 4, negativeSegments: 6 957: positiveSegments: 4, negativeSegments: 9 958: positiveSegments: 7, negativeSegments: 0 959: positiveSegments: 0, negativeSegments: 9 963: positiveSegments: 8, negativeSegments: 12 967: positiveSegments: 0, negativeSegments: 3 969: positiveSegments: 6, negativeSegments: 0 971: positiveSegments: 7, negativeSegments: 9 972: positiveSegments: 12, negativeSegments: 0 973: positiveSegments: 6, negativeSegments: 0 976: positiveSegments: 0, negativeSegments: 15 977: positiveSegments: 6, negativeSegments: 3 979: positiveSegments: 10, negativeSegments: 0 980: positiveSegments: 12, negativeSegments: 15 983: positiveSegments: 0, negativeSegments: 9 984: positiveSegments: 8, negativeSegments: 0 985: positiveSegments: 16, negativeSegments: 6 986: positiveSegments: 0, negativeSegments: 9 988: positiveSegments: 5, negativeSegments: 9 990: positiveSegments: 13, negativeSegments: 0 991: positiveSegments: 11, negativeSegments: 0 992: positiveSegments: 4, negativeSegments: 12 994: positiveSegments: 6, negativeSegments: 0 995: positiveSegments: 12, negativeSegments: 3 1002: positiveSegments: 8, negativeSegments: 6 1003: positiveSegments: 2, negativeSegments: 3 1005: positiveSegments: 4, negativeSegments: 12 1012: positiveSegments: 9, negativeSegments: 9 1013: positiveSegments: 19, negativeSegments: 9 1015: positiveSegments: 1, negativeSegments: 0 1016: positiveSegments: 5, negativeSegments: 6 1017: positiveSegments: 3, negativeSegments: 3 1018: positiveSegments: 16, negativeSegments: 12 1020: positiveSegments: 4, negativeSegments: 6 1022: positiveSegments: 0, negativeSegments: 9 1024: positiveSegments: 6, negativeSegments: 0 1025: positiveSegments: 10, negativeSegments: 15 1026: positiveSegments: 16, negativeSegments: 12 1027: positiveSegments: 16, negativeSegments: 3 1028: positiveSegments: 0, negativeSegments: 3 1029: positiveSegments: 2, negativeSegments: 9 1030: positiveSegments: 2, negativeSegments: 6 1032: positiveSegments: 11, negativeSegments: 0 1033: positiveSegments: 4, negativeSegments: 3 1034: exit early, no segments to save 1034: positiveSegments: 0, negativeSegments: 0 1035: positiveSegments: 2, negativeSegments: 9 1037: positiveSegments: 1, negativeSegments: 21 1038: positiveSegments: 0, negativeSegments: 6 1040: positiveSegments: 8, negativeSegments: 3 1041: positiveSegments: 4, negativeSegments: 6 1043: positiveSegments: 0, negativeSegments: 3 1044: positiveSegments: 8, negativeSegments: 9 count processed: 500, current case index: 1046 1046: positiveSegments: 11, negativeSegments: 0 1047: positiveSegments: 0, negativeSegments: 6 1049: positiveSegments: 4, negativeSegments: 6 1050: positiveSegments: 7, negativeSegments: 3 1051: positiveSegments: 4, negativeSegments: 3 1055: exit early, no segments to save 1055: positiveSegments: 0, negativeSegments: 0 1056: positiveSegments: 10, negativeSegments: 0 1061: positiveSegments: 0, negativeSegments: 6 1063: positiveSegments: 14, negativeSegments: 3 1065: positiveSegments: 8, negativeSegments: 0 1069: positiveSegments: 32, negativeSegments: 3 1073: positiveSegments: 11, negativeSegments: 0 1074: positiveSegments: 13, negativeSegments: 0 1076: positiveSegments: 18, negativeSegments: 3 1077: positiveSegments: 4, negativeSegments: 0 1078: positiveSegments: 8, negativeSegments: 3 1081: positiveSegments: 7, negativeSegments: 0 1083: positiveSegments: 10, negativeSegments: 0 1084: positiveSegments: 4, negativeSegments: 6 1086: positiveSegments: 21, negativeSegments: 6 1087: positiveSegments: 13, negativeSegments: 6 1088: positiveSegments: 0, negativeSegments: 6 1089: positiveSegments: 4, negativeSegments: 3 1090: positiveSegments: 10, negativeSegments: 3 1091: positiveSegments: 1, negativeSegments: 0 1093: positiveSegments: 12, negativeSegments: 6 1094: positiveSegments: 17, negativeSegments: 18 1095: positiveSegments: 16, negativeSegments: 18 1096: positiveSegments: 6, negativeSegments: 12 1097: positiveSegments: 14, negativeSegments: 3 1098: positiveSegments: 8, negativeSegments: 0 1102: positiveSegments: 7, negativeSegments: 0 1108: positiveSegments: 8, negativeSegments: 3 1109: positiveSegments: 0, negativeSegments: 3 1113: positiveSegments: 8, negativeSegments: 12 1114: positiveSegments: 20, negativeSegments: 0 1115: positiveSegments: 10, negativeSegments: 12 1118: positiveSegments: 16, negativeSegments: 12 1123: positiveSegments: 17, negativeSegments: 6 1124: positiveSegments: 9, negativeSegments: 0 1125: positiveSegments: 12, negativeSegments: 12 1127: positiveSegments: 10, negativeSegments: 3 1131: positiveSegments: 0, negativeSegments: 6 1132: positiveSegments: 27, negativeSegments: 6 1133: positiveSegments: 3, negativeSegments: 0 1135: positiveSegments: 0, negativeSegments: 6 1138: positiveSegments: 8, negativeSegments: 0 1139: positiveSegments: 0, negativeSegments: 3 1143: positiveSegments: 7, negativeSegments: 0 1144: positiveSegments: 5, negativeSegments: 0 1145: positiveSegments: 4, negativeSegments: 12 1154: positiveSegments: 0, negativeSegments: 6 1156: exit early, no segments to save 1156: positiveSegments: 0, negativeSegments: 0 1158: positiveSegments: 4, negativeSegments: 0 1159: positiveSegments: 10, negativeSegments: 3 1160: positiveSegments: 17, negativeSegments: 0 1165: positiveSegments: 22, negativeSegments: 0 1166: positiveSegments: 15, negativeSegments: 3 1170: positiveSegments: 0, negativeSegments: 6 1174: positiveSegments: 4, negativeSegments: 0 1176: positiveSegments: 0, negativeSegments: 6 1180: positiveSegments: 10, negativeSegments: 9 1181: positiveSegments: 12, negativeSegments: 0 1182: positiveSegments: 8, negativeSegments: 9 1184: positiveSegments: 10, negativeSegments: 0 1185: positiveSegments: 35, negativeSegments: 6 1186: positiveSegments: 0, negativeSegments: 3 1187: positiveSegments: 2, negativeSegments: 3 1189: positiveSegments: 17, negativeSegments: 0 1190: exit early, no segments to save 1190: positiveSegments: 0, negativeSegments: 0 1191: positiveSegments: 47, negativeSegments: 6 1193: positiveSegments: 26, negativeSegments: 3 1194: positiveSegments: 20, negativeSegments: 3 1196: positiveSegments: 4, negativeSegments: 0 1199: positiveSegments: 4, negativeSegments: 3 1200: positiveSegments: 11, negativeSegments: 6 1201: positiveSegments: 0, negativeSegments: 6 1202: positiveSegments: 0, negativeSegments: 3 1203: positiveSegments: 2, negativeSegments: 0 1204: positiveSegments: 19, negativeSegments: 3 1205: positiveSegments: 14, negativeSegments: 3 1208: positiveSegments: 8, negativeSegments: 0 1209: positiveSegments: 7, negativeSegments: 0 1212: positiveSegments: 0, negativeSegments: 3 1213: positiveSegments: 1, negativeSegments: 6 1215: positiveSegments: 28, negativeSegments: 3 1216: positiveSegments: 21, negativeSegments: 21 1217: positiveSegments: 9, negativeSegments: 0 1219: positiveSegments: 16, negativeSegments: 6 1221: positiveSegments: 10, negativeSegments: 6 1222: positiveSegments: 4, negativeSegments: 9 1224: positiveSegments: 12, negativeSegments: 0 1225: positiveSegments: 8, negativeSegments: 3 1228: positiveSegments: 15, negativeSegments: 0 1229: positiveSegments: 26, negativeSegments: 0 1230: positiveSegments: 14, negativeSegments: 15 1231: positiveSegments: 3, negativeSegments: 0 1232: positiveSegments: 3, negativeSegments: 6 1233: positiveSegments: 9, negativeSegments: 0 1234: positiveSegments: 0, negativeSegments: 3 count processed: 600, current case index: 1236 1236: positiveSegments: 16, negativeSegments: 0 1237: positiveSegments: 6, negativeSegments: 0 1238: exit early, no segments to save 1238: positiveSegments: 0, negativeSegments: 0 1239: positiveSegments: 2, negativeSegments: 12 1240: positiveSegments: 8, negativeSegments: 6 1244: positiveSegments: 11, negativeSegments: 12 1248: positiveSegments: 4, negativeSegments: 12 1249: positiveSegments: 5, negativeSegments: 6 1256: positiveSegments: 0, negativeSegments: 6 1261: positiveSegments: 14, negativeSegments: 6 1263: positiveSegments: 11, negativeSegments: 9 1264: exit early, no segments to save 1264: positiveSegments: 0, negativeSegments: 0 1265: positiveSegments: 0, negativeSegments: 9 1267: positiveSegments: 0, negativeSegments: 6 1268: positiveSegments: 0, negativeSegments: 6 1271: exit early, no segments to save 1271: positiveSegments: 0, negativeSegments: 0 1272: positiveSegments: 15, negativeSegments: 0 1275: positiveSegments: 26, negativeSegments: 3 1277: positiveSegments: 4, negativeSegments: 9 1279: positiveSegments: 0, negativeSegments: 6 1280: positiveSegments: 0, negativeSegments: 6 1283: positiveSegments: 5, negativeSegments: 0 1285: positiveSegments: 7, negativeSegments: 6 1286: positiveSegments: 27, negativeSegments: 15 1290: positiveSegments: 0, negativeSegments: 3 1291: positiveSegments: 19, negativeSegments: 6 1292: positiveSegments: 25, negativeSegments: 6 1293: positiveSegments: 17, negativeSegments: 0 1294: positiveSegments: 4, negativeSegments: 0 1297: positiveSegments: 7, negativeSegments: 0 1298: positiveSegments: 4, negativeSegments: 6 1300: positiveSegments: 9, negativeSegments: 0 1301: exit early, no segments to save 1301: positiveSegments: 0, negativeSegments: 0 1302: positiveSegments: 0, negativeSegments: 12 1303: positiveSegments: 4, negativeSegments: 9 1305: positiveSegments: 18, negativeSegments: 0 1307: positiveSegments: 14, negativeSegments: 12 1309: positiveSegments: 13, negativeSegments: 9 1311: positiveSegments: 0, negativeSegments: 3 1313: positiveSegments: 11, negativeSegments: 6 1315: positiveSegments: 10, negativeSegments: 6 1316: positiveSegments: 14, negativeSegments: 3 1317: positiveSegments: 4, negativeSegments: 0 1319: positiveSegments: 4, negativeSegments: 3 1320: positiveSegments: 6, negativeSegments: 0 1321: positiveSegments: 2, negativeSegments: 15 1323: positiveSegments: 8, negativeSegments: 0 1324: positiveSegments: 6, negativeSegments: 9 1325: positiveSegments: 20, negativeSegments: 9 1333: positiveSegments: 2, negativeSegments: 6 1335: positiveSegments: 22, negativeSegments: 9 1339: positiveSegments: 2, negativeSegments: 3 1340: positiveSegments: 0, negativeSegments: 3 1341: positiveSegments: 4, negativeSegments: 3 1343: positiveSegments: 10, negativeSegments: 3 1344: positiveSegments: 0, negativeSegments: 9 1346: positiveSegments: 0, negativeSegments: 9 1347: positiveSegments: 0, negativeSegments: 9 1350: positiveSegments: 2, negativeSegments: 15 1353: positiveSegments: 9, negativeSegments: 0 1356: positiveSegments: 7, negativeSegments: 0 1358: positiveSegments: 3, negativeSegments: 9 1359: positiveSegments: 27, negativeSegments: 6 1362: positiveSegments: 9, negativeSegments: 3 1364: positiveSegments: 18, negativeSegments: 0 1365: positiveSegments: 8, negativeSegments: 0 1367: positiveSegments: 23, negativeSegments: 3 1368: positiveSegments: 4, negativeSegments: 0 1369: exit early, no segments to save 1369: positiveSegments: 0, negativeSegments: 0 1374: positiveSegments: 20, negativeSegments: 15 1375: positiveSegments: 11, negativeSegments: 3 1376: positiveSegments: 4, negativeSegments: 3 1381: positiveSegments: 13, negativeSegments: 6 1383: positiveSegments: 10, negativeSegments: 9 1386: positiveSegments: 0, negativeSegments: 3 1389: positiveSegments: 4, negativeSegments: 9 1392: positiveSegments: 3, negativeSegments: 0 1396: positiveSegments: 8, negativeSegments: 9 1397: positiveSegments: 13, negativeSegments: 3 1398: positiveSegments: 3, negativeSegments: 6 1399: positiveSegments: 19, negativeSegments: 3 1402: positiveSegments: 14, negativeSegments: 0 1403: positiveSegments: 22, negativeSegments: 24 1404: positiveSegments: 0, negativeSegments: 3 1407: positiveSegments: 36, negativeSegments: 3 1408: positiveSegments: 0, negativeSegments: 3 1413: exit early, no segments to save 1413: positiveSegments: 0, negativeSegments: 0 1414: positiveSegments: 10, negativeSegments: 0 1415: positiveSegments: 6, negativeSegments: 3 1416: positiveSegments: 23, negativeSegments: 3 1417: positiveSegments: 3, negativeSegments: 3 1421: positiveSegments: 8, negativeSegments: 6 1422: positiveSegments: 2, negativeSegments: 15 1426: positiveSegments: 4, negativeSegments: 6 1428: positiveSegments: 5, negativeSegments: 6 1432: positiveSegments: 16, negativeSegments: 6 1434: positiveSegments: 24, negativeSegments: 3 1436: positiveSegments: 8, negativeSegments: 12 1438: positiveSegments: 9, negativeSegments: 0 1439: positiveSegments: 10, negativeSegments: 3 count processed: 700, current case index: 1440 1440: exit early, no segments to save 1440: positiveSegments: 0, negativeSegments: 0 1442: positiveSegments: 6, negativeSegments: 12 1444: positiveSegments: 4, negativeSegments: 0 1446: positiveSegments: 9, negativeSegments: 3 1451: exit early, no segments to save 1451: positiveSegments: 0, negativeSegments: 0 1452: positiveSegments: 0, negativeSegments: 6 1454: positiveSegments: 0, negativeSegments: 18 1456: positiveSegments: 4, negativeSegments: 0 1458: positiveSegments: 12, negativeSegments: 0 1463: positiveSegments: 2, negativeSegments: 9 1465: positiveSegments: 8, negativeSegments: 3 1468: positiveSegments: 11, negativeSegments: 18 1469: positiveSegments: 30, negativeSegments: 3 1470: positiveSegments: 7, negativeSegments: 0 1471: positiveSegments: 4, negativeSegments: 0 1473: positiveSegments: 4, negativeSegments: 6 1474: positiveSegments: 3, negativeSegments: 6 1475: positiveSegments: 41, negativeSegments: 3 1478: positiveSegments: 7, negativeSegments: 6 1479: positiveSegments: 4, negativeSegments: 3 1482: positiveSegments: 0, negativeSegments: 18 1485: positiveSegments: 26, negativeSegments: 0 1486: positiveSegments: 22, negativeSegments: 0 1487: positiveSegments: 0, negativeSegments: 6 1488: positiveSegments: 12, negativeSegments: 0 1489: positiveSegments: 15, negativeSegments: 3 1490: positiveSegments: 11, negativeSegments: 3 1492: positiveSegments: 43, negativeSegments: 0 1493: positiveSegments: 13, negativeSegments: 9 1496: positiveSegments: 4, negativeSegments: 6 1497: positiveSegments: 16, negativeSegments: 6 1498: positiveSegments: 0, negativeSegments: 6 1500: positiveSegments: 5, negativeSegments: 6 1503: positiveSegments: 7, negativeSegments: 0 1505: exit early, no segments to save 1505: positiveSegments: 0, negativeSegments: 0 1512: positiveSegments: 4, negativeSegments: 3 1515: positiveSegments: 38, negativeSegments: 3 1520: positiveSegments: 18, negativeSegments: 0 1521: positiveSegments: 16, negativeSegments: 9 1522: positiveSegments: 4, negativeSegments: 9 1523: positiveSegments: 11, negativeSegments: 27 1525: positiveSegments: 22, negativeSegments: 3 1526: positiveSegments: 0, negativeSegments: 9 1527: positiveSegments: 0, negativeSegments: 6 1536: positiveSegments: 0, negativeSegments: 6 1537: positiveSegments: 12, negativeSegments: 0 1539: positiveSegments: 7, negativeSegments: 24 1540: positiveSegments: 6, negativeSegments: 0 1541: positiveSegments: 13, negativeSegments: 0 1542: positiveSegments: 16, negativeSegments: 6 1545: positiveSegments: 19, negativeSegments: 3 1546: positiveSegments: 4, negativeSegments: 3 1548: positiveSegments: 0, negativeSegments: 12 1549: positiveSegments: 7, negativeSegments: 6 1552: positiveSegments: 5, negativeSegments: 0 1553: positiveSegments: 5, negativeSegments: 0 1554: exit early, no segments to save 1554: positiveSegments: 0, negativeSegments: 0 1555: exit early, no segments to save 1555: positiveSegments: 0, negativeSegments: 0 1556: positiveSegments: 5, negativeSegments: 18 1558: positiveSegments: 18, negativeSegments: 0 1559: positiveSegments: 14, negativeSegments: 6 1561: positiveSegments: 11, negativeSegments: 3 1562: positiveSegments: 6, negativeSegments: 9 1563: positiveSegments: 4, negativeSegments: 0 1564: positiveSegments: 30, negativeSegments: 3 1566: positiveSegments: 8, negativeSegments: 6 1567: positiveSegments: 11, negativeSegments: 0 1568: positiveSegments: 16, negativeSegments: 0 1574: positiveSegments: 4, negativeSegments: 12 1575: positiveSegments: 8, negativeSegments: 6 1580: positiveSegments: 20, negativeSegments: 3 1581: positiveSegments: 4, negativeSegments: 9 1583: positiveSegments: 20, negativeSegments: 6 1585: exit early, no segments to save 1585: positiveSegments: 0, negativeSegments: 0 1586: positiveSegments: 4, negativeSegments: 9 1590: positiveSegments: 20, negativeSegments: 0 1591: positiveSegments: 9, negativeSegments: 6 1594: positiveSegments: 4, negativeSegments: 0 1595: positiveSegments: 8, negativeSegments: 12 1596: exit early, no segments to save 1596: positiveSegments: 0, negativeSegments: 0 1597: positiveSegments: 8, negativeSegments: 18 1599: positiveSegments: 13, negativeSegments: 15 1600: positiveSegments: 4, negativeSegments: 0 1602: positiveSegments: 6, negativeSegments: 18 1605: positiveSegments: 17, negativeSegments: 12 1608: positiveSegments: 8, negativeSegments: 0 1610: positiveSegments: 0, negativeSegments: 3 1611: exit early, no segments to save 1611: positiveSegments: 0, negativeSegments: 0 1612: exit early, no segments to save 1612: positiveSegments: 0, negativeSegments: 0 1613: positiveSegments: 4, negativeSegments: 6 1614: positiveSegments: 0, negativeSegments: 9 1615: positiveSegments: 11, negativeSegments: 12 1616: positiveSegments: 12, negativeSegments: 6 1618: positiveSegments: 4, negativeSegments: 0 1620: positiveSegments: 14, negativeSegments: 0 1623: positiveSegments: 6, negativeSegments: 0 1630: positiveSegments: 15, negativeSegments: 3 1632: positiveSegments: 7, negativeSegments: 0 1633: positiveSegments: 18, negativeSegments: 3 1636: positiveSegments: 7, negativeSegments: 3 count processed: 800, current case index: 1639 1639: positiveSegments: 13, negativeSegments: 0 1641: positiveSegments: 13, negativeSegments: 9 1642: positiveSegments: 9, negativeSegments: 0 1647: positiveSegments: 8, negativeSegments: 3 1648: positiveSegments: 10, negativeSegments: 3 1656: positiveSegments: 0, negativeSegments: 3 1657: positiveSegments: 0, negativeSegments: 12 1658: positiveSegments: 8, negativeSegments: 6 1662: exit early, no segments to save 1662: positiveSegments: 0, negativeSegments: 0 1665: positiveSegments: 15, negativeSegments: 0 1666: positiveSegments: 12, negativeSegments: 9 1668: positiveSegments: 4, negativeSegments: 0 1671: positiveSegments: 19, negativeSegments: 3 1672: positiveSegments: 4, negativeSegments: 9 1673: positiveSegments: 10, negativeSegments: 0 1674: positiveSegments: 18, negativeSegments: 0 1675: nothing saved, all segments filtered 1675: positiveSegments: 3, negativeSegments: 0 1676: exit early, no segments to save 1676: positiveSegments: 0, negativeSegments: 0 1681: positiveSegments: 4, negativeSegments: 0 1684: positiveSegments: 8, negativeSegments: 0 1685: positiveSegments: 0, negativeSegments: 3 1687: positiveSegments: 0, negativeSegments: 3 1688: positiveSegments: 6, negativeSegments: 0 1689: positiveSegments: 3, negativeSegments: 6 1690: positiveSegments: 10, negativeSegments: 6 1694: positiveSegments: 15, negativeSegments: 3 1695: positiveSegments: 4, negativeSegments: 3 1696: positiveSegments: 0, negativeSegments: 18 1697: positiveSegments: 3, negativeSegments: 0 1699: positiveSegments: 0, negativeSegments: 3 1700: positiveSegments: 10, negativeSegments: 6 1703: positiveSegments: 38, negativeSegments: 12 1705: positiveSegments: 4, negativeSegments: 12 1706: positiveSegments: 3, negativeSegments: 6 1708: positiveSegments: 17, negativeSegments: 12 1710: positiveSegments: 14, negativeSegments: 3 1714: positiveSegments: 6, negativeSegments: 9 1716: positiveSegments: 23, negativeSegments: 0 1718: positiveSegments: 27, negativeSegments: 3 1719: positiveSegments: 4, negativeSegments: 0 1722: positiveSegments: 0, negativeSegments: 18 1724: positiveSegments: 9, negativeSegments: 3 1726: positiveSegments: 6, negativeSegments: 9 1728: positiveSegments: 0, negativeSegments: 9 1729: positiveSegments: 16, negativeSegments: 15 1730: positiveSegments: 14, negativeSegments: 0 1732: positiveSegments: 8, negativeSegments: 6 1733: positiveSegments: 18, negativeSegments: 0 1735: positiveSegments: 4, negativeSegments: 6 1737: positiveSegments: 12, negativeSegments: 6 1738: positiveSegments: 19, negativeSegments: 0 1743: positiveSegments: 17, negativeSegments: 9 1745: positiveSegments: 27, negativeSegments: 6 1747: positiveSegments: 0, negativeSegments: 9 1748: positiveSegments: 5, negativeSegments: 3 1749: positiveSegments: 0, negativeSegments: 6 1752: positiveSegments: 42, negativeSegments: 3 1753: positiveSegments: 6, negativeSegments: 0 1756: positiveSegments: 4, negativeSegments: 6 1757: positiveSegments: 0, negativeSegments: 6 1759: positiveSegments: 2, negativeSegments: 0 1761: positiveSegments: 26, negativeSegments: 0 1762: positiveSegments: 0, negativeSegments: 9 1763: positiveSegments: 5, negativeSegments: 0 1765: positiveSegments: 10, negativeSegments: 6 1766: positiveSegments: 16, negativeSegments: 0 1768: positiveSegments: 8, negativeSegments: 0 1771: positiveSegments: 9, negativeSegments: 0 1773: positiveSegments: 6, negativeSegments: 0 1775: positiveSegments: 11, negativeSegments: 0 1777: positiveSegments: 0, negativeSegments: 12 1779: positiveSegments: 2, negativeSegments: 12 1783: positiveSegments: 29, negativeSegments: 3 1784: positiveSegments: 12, negativeSegments: 0 1785: positiveSegments: 25, negativeSegments: 0 1793: positiveSegments: 13, negativeSegments: 0 1799: positiveSegments: 41, negativeSegments: 3 1800: positiveSegments: 0, negativeSegments: 3 1802: positiveSegments: 4, negativeSegments: 6 1803: positiveSegments: 62, negativeSegments: 21 1805: positiveSegments: 8, negativeSegments: 0 1809: positiveSegments: 3, negativeSegments: 12 1810: positiveSegments: 2, negativeSegments: 0 1812: positiveSegments: 15, negativeSegments: 0 1814: positiveSegments: 6, negativeSegments: 3 1816: positiveSegments: 18, negativeSegments: 15 1819: positiveSegments: 21, negativeSegments: 0 1820: positiveSegments: 17, negativeSegments: 0 1822: positiveSegments: 9, negativeSegments: 3 1823: positiveSegments: 4, negativeSegments: 6 1825: positiveSegments: 0, negativeSegments: 3 1826: positiveSegments: 6, negativeSegments: 0 1832: positiveSegments: 32, negativeSegments: 3 1833: positiveSegments: 30, negativeSegments: 0 1834: positiveSegments: 15, negativeSegments: 0 1835: positiveSegments: 23, negativeSegments: 0 1836: positiveSegments: 8, negativeSegments: 0 1837: positiveSegments: 2, negativeSegments: 3 1838: positiveSegments: 12, negativeSegments: 6 1840: positiveSegments: 5, negativeSegments: 3 count processed: 900, current case index: 1843 1843: positiveSegments: 17, negativeSegments: 6 1844: positiveSegments: 12, negativeSegments: 0 1846: positiveSegments: 5, negativeSegments: 0 1848: positiveSegments: 10, negativeSegments: 6 1852: positiveSegments: 22, negativeSegments: 3 1853: positiveSegments: 2, negativeSegments: 9 1854: positiveSegments: 16, negativeSegments: 0 1855: positiveSegments: 28, negativeSegments: 0 1862: positiveSegments: 19, negativeSegments: 9 1864: exit early, no segments to save 1864: positiveSegments: 0, negativeSegments: 0 1865: positiveSegments: 7, negativeSegments: 0 1866: positiveSegments: 4, negativeSegments: 0 1869: positiveSegments: 0, negativeSegments: 3 1872: positiveSegments: 15, negativeSegments: 0 1873: positiveSegments: 16, negativeSegments: 9 1874: positiveSegments: 8, negativeSegments: 0 1875: exit early, no segments to save 1875: positiveSegments: 0, negativeSegments: 0 1876: exit early, no segments to save 1876: positiveSegments: 0, negativeSegments: 0 1882: positiveSegments: 0, negativeSegments: 3 1884: positiveSegments: 17, negativeSegments: 12 1885: positiveSegments: 0, negativeSegments: 3 1886: positiveSegments: 10, negativeSegments: 6 1888: positiveSegments: 14, negativeSegments: 3 1891: positiveSegments: 0, negativeSegments: 6 1892: positiveSegments: 16, negativeSegments: 9 1893: positiveSegments: 20, negativeSegments: 12 1894: positiveSegments: 8, negativeSegments: 0 1896: positiveSegments: 6, negativeSegments: 3 1898: positiveSegments: 10, negativeSegments: 0 1899: positiveSegments: 0, negativeSegments: 6 1900: positiveSegments: 32, negativeSegments: 3 1901: positiveSegments: 17, negativeSegments: 6 1903: positiveSegments: 18, negativeSegments: 6 1907: positiveSegments: 4, negativeSegments: 0 1910: positiveSegments: 3, negativeSegments: 0 1912: positiveSegments: 10, negativeSegments: 21 1913: positiveSegments: 15, negativeSegments: 0 1914: positiveSegments: 11, negativeSegments: 3 1915: positiveSegments: 3, negativeSegments: 0 1916: positiveSegments: 0, negativeSegments: 15 1918: positiveSegments: 17, negativeSegments: 9 1920: positiveSegments: 12, negativeSegments: 3 1922: positiveSegments: 4, negativeSegments: 9 1925: positiveSegments: 0, negativeSegments: 3 1926: positiveSegments: 4, negativeSegments: 0 1928: positiveSegments: 28, negativeSegments: 0 1931: positiveSegments: 7, negativeSegments: 0 1932: positiveSegments: 29, negativeSegments: 6 1933: positiveSegments: 4, negativeSegments: 0 1934: positiveSegments: 26, negativeSegments: 0 1935: positiveSegments: 19, negativeSegments: 12 1936: positiveSegments: 65, negativeSegments: 3 1937: positiveSegments: 12, negativeSegments: 9 1938: positiveSegments: 12, negativeSegments: 15 1941: positiveSegments: 38, negativeSegments: 6 1942: positiveSegments: 6, negativeSegments: 0 1943: exit early, no segments to save 1943: positiveSegments: 0, negativeSegments: 0 1944: positiveSegments: 11, negativeSegments: 3 1947: positiveSegments: 4, negativeSegments: 6 1949: positiveSegments: 8, negativeSegments: 6 1950: positiveSegments: 0, negativeSegments: 6 1955: positiveSegments: 7, negativeSegments: 6 1956: positiveSegments: 7, negativeSegments: 3 1957: positiveSegments: 7, negativeSegments: 0 1959: positiveSegments: 19, negativeSegments: 3 1961: positiveSegments: 23, negativeSegments: 12 1963: positiveSegments: 21, negativeSegments: 9 1965: positiveSegments: 15, negativeSegments: 0 1966: positiveSegments: 8, negativeSegments: 0 1969: positiveSegments: 3, negativeSegments: 6 1973: positiveSegments: 1, negativeSegments: 9 1976: positiveSegments: 21, negativeSegments: 12 1978: positiveSegments: 16, negativeSegments: 6 1985: positiveSegments: 8, negativeSegments: 0 1986: positiveSegments: 2, negativeSegments: 0 1988: positiveSegments: 2, negativeSegments: 6 1993: positiveSegments: 1, negativeSegments: 6 1994: positiveSegments: 4, negativeSegments: 3 1995: positiveSegments: 12, negativeSegments: 0 1996: positiveSegments: 12, negativeSegments: 3 2000: positiveSegments: 0, negativeSegments: 6 2002: positiveSegments: 7, negativeSegments: 6 2004: positiveSegments: 6, negativeSegments: 6 2009: exit early, no segments to save 2009: positiveSegments: 0, negativeSegments: 0 2010: positiveSegments: 8, negativeSegments: 3 2011: positiveSegments: 4, negativeSegments: 18 2012: positiveSegments: 3, negativeSegments: 0 2014: positiveSegments: 7, negativeSegments: 9 2017: positiveSegments: 13, negativeSegments: 0 2018: positiveSegments: 22, negativeSegments: 3 2020: positiveSegments: 11, negativeSegments: 0 2025: positiveSegments: 6, negativeSegments: 0 2026: positiveSegments: 13, negativeSegments: 0 2028: positiveSegments: 1, negativeSegments: 21 2029: positiveSegments: 0, negativeSegments: 9 2034: exit early, no segments to save 2034: positiveSegments: 0, negativeSegments: 0 2040: positiveSegments: 10, negativeSegments: 6 2041: positiveSegments: 19, negativeSegments: 3 2044: positiveSegments: 8, negativeSegments: 6 2046: positiveSegments: 4, negativeSegments: 0 count processed: 1000, current case index: 2049 2049: positiveSegments: 4, negativeSegments: 3 2051: exit early, no segments to save 2051: positiveSegments: 0, negativeSegments: 0 2055: positiveSegments: 12, negativeSegments: 12 2057: positiveSegments: 16, negativeSegments: 0 2058: positiveSegments: 8, negativeSegments: 0 2060: positiveSegments: 4, negativeSegments: 15 2061: positiveSegments: 23, negativeSegments: 0 2062: positiveSegments: 0, negativeSegments: 12 2064: positiveSegments: 16, negativeSegments: 0 2066: positiveSegments: 11, negativeSegments: 6 2067: positiveSegments: 19, negativeSegments: 3 2068: positiveSegments: 14, negativeSegments: 0 2072: positiveSegments: 4, negativeSegments: 6 2074: exit early, no segments to save 2074: positiveSegments: 0, negativeSegments: 0 2075: positiveSegments: 0, negativeSegments: 3 2081: positiveSegments: 2, negativeSegments: 0 2082: positiveSegments: 11, negativeSegments: 9 2086: positiveSegments: 10, negativeSegments: 0 2088: positiveSegments: 0, negativeSegments: 6 2097: positiveSegments: 2, negativeSegments: 0 2098: positiveSegments: 0, negativeSegments: 3 2106: positiveSegments: 0, negativeSegments: 15 2111: positiveSegments: 2, negativeSegments: 0 2112: positiveSegments: 8, negativeSegments: 0 2114: positiveSegments: 4, negativeSegments: 3 2117: positiveSegments: 0, negativeSegments: 3 2118: positiveSegments: 0, negativeSegments: 12 2121: positiveSegments: 8, negativeSegments: 3 2130: positiveSegments: 0, negativeSegments: 9 2132: positiveSegments: 7, negativeSegments: 0 2133: positiveSegments: 9, negativeSegments: 0 2135: exit early, no segments to save 2135: positiveSegments: 0, negativeSegments: 0 2136: positiveSegments: 25, negativeSegments: 9 2139: positiveSegments: 0, negativeSegments: 6 2142: positiveSegments: 8, negativeSegments: 0 2147: positiveSegments: 17, negativeSegments: 9 2148: positiveSegments: 4, negativeSegments: 6 2149: positiveSegments: 4, negativeSegments: 6 2150: positiveSegments: 4, negativeSegments: 6 2153: positiveSegments: 41, negativeSegments: 12 2154: positiveSegments: 0, negativeSegments: 9 2157: exit early, no segments to save 2157: positiveSegments: 0, negativeSegments: 0 2158: positiveSegments: 0, negativeSegments: 3 2161: positiveSegments: 20, negativeSegments: 3 2162: nothing saved, all segments filtered 2162: positiveSegments: 2, negativeSegments: 0 2163: positiveSegments: 4, negativeSegments: 3 2165: positiveSegments: 22, negativeSegments: 12 2168: positiveSegments: 31, negativeSegments: 3 2169: positiveSegments: 2, negativeSegments: 9 2170: exit early, no segments to save 2170: positiveSegments: 0, negativeSegments: 0 2172: positiveSegments: 6, negativeSegments: 18 2174: exit early, no segments to save 2174: positiveSegments: 0, negativeSegments: 0 2175: positiveSegments: 0, negativeSegments: 3 2176: positiveSegments: 8, negativeSegments: 3 2178: exit early, no segments to save 2178: positiveSegments: 0, negativeSegments: 0 2183: positiveSegments: 6, negativeSegments: 9 2185: positiveSegments: 8, negativeSegments: 12 2187: positiveSegments: 7, negativeSegments: 0 2192: positiveSegments: 19, negativeSegments: 9 2194: positiveSegments: 3, negativeSegments: 18 2195: positiveSegments: 8, negativeSegments: 6 2196: positiveSegments: 4, negativeSegments: 3 2197: positiveSegments: 25, negativeSegments: 6 2201: positiveSegments: 21, negativeSegments: 0 2205: positiveSegments: 6, negativeSegments: 3 2206: positiveSegments: 8, negativeSegments: 3 2210: positiveSegments: 12, negativeSegments: 3 2213: positiveSegments: 0, negativeSegments: 3 2214: positiveSegments: 10, negativeSegments: 3 2218: exit early, no segments to save 2218: positiveSegments: 0, negativeSegments: 0 2219: nothing saved, all segments filtered 2219: positiveSegments: 13, negativeSegments: 0 2221: positiveSegments: 4, negativeSegments: 0 2222: positiveSegments: 6, negativeSegments: 0 2223: positiveSegments: 4, negativeSegments: 0 2224: positiveSegments: 5, negativeSegments: 0 2225: positiveSegments: 15, negativeSegments: 12 2229: positiveSegments: 0, negativeSegments: 9 2231: positiveSegments: 14, negativeSegments: 9 2236: positiveSegments: 17, negativeSegments: 3 2238: positiveSegments: 2, negativeSegments: 24 2241: positiveSegments: 0, negativeSegments: 9 2242: positiveSegments: 6, negativeSegments: 6 2243: positiveSegments: 20, negativeSegments: 3 2244: positiveSegments: 0, negativeSegments: 6 2246: positiveSegments: 8, negativeSegments: 3 2248: positiveSegments: 3, negativeSegments: 3 2249: positiveSegments: 4, negativeSegments: 0 2251: positiveSegments: 3, negativeSegments: 0 2252: positiveSegments: 14, negativeSegments: 9 2253: nothing saved, all segments filtered 2253: positiveSegments: 1, negativeSegments: 0 2255: positiveSegments: 0, negativeSegments: 3 2258: positiveSegments: 4, negativeSegments: 3 2261: positiveSegments: 18, negativeSegments: 0 2265: positiveSegments: 14, negativeSegments: 0 2267: positiveSegments: 18, negativeSegments: 3 2272: positiveSegments: 15, negativeSegments: 0 2273: positiveSegments: 0, negativeSegments: 12 2275: positiveSegments: 14, negativeSegments: 3 2279: positiveSegments: 8, negativeSegments: 0 2280: positiveSegments: 11, negativeSegments: 6 count processed: 1100, current case index: 2281 2281: exit early, no segments to save 2281: positiveSegments: 0, negativeSegments: 0 2282: positiveSegments: 0, negativeSegments: 3 2283: positiveSegments: 0, negativeSegments: 3 2284: positiveSegments: 7, negativeSegments: 3 2291: positiveSegments: 4, negativeSegments: 0 2295: positiveSegments: 5, negativeSegments: 0 2296: positiveSegments: 4, negativeSegments: 9 2298: positiveSegments: 4, negativeSegments: 3 2299: positiveSegments: 4, negativeSegments: 3 2300: positiveSegments: 8, negativeSegments: 21 2302: positiveSegments: 4, negativeSegments: 9 2304: positiveSegments: 13, negativeSegments: 6 2305: positiveSegments: 3, negativeSegments: 6 2306: positiveSegments: 11, negativeSegments: 6 2307: positiveSegments: 4, negativeSegments: 12 2309: positiveSegments: 11, negativeSegments: 0 2310: positiveSegments: 6, negativeSegments: 15 2311: positiveSegments: 14, negativeSegments: 0 2313: positiveSegments: 2, negativeSegments: 0 2315: positiveSegments: 9, negativeSegments: 0 2317: positiveSegments: 4, negativeSegments: 0 2318: positiveSegments: 31, negativeSegments: 0 2319: positiveSegments: 3, negativeSegments: 3 2321: positiveSegments: 4, negativeSegments: 3 2324: positiveSegments: 18, negativeSegments: 3 2325: positiveSegments: 17, negativeSegments: 3 2326: positiveSegments: 35, negativeSegments: 0 2327: positiveSegments: 11, negativeSegments: 0 2331: positiveSegments: 11, negativeSegments: 0 2332: positiveSegments: 34, negativeSegments: 6 2333: positiveSegments: 4, negativeSegments: 3 2334: positiveSegments: 21, negativeSegments: 3 2335: positiveSegments: 8, negativeSegments: 0 2336: positiveSegments: 30, negativeSegments: 15 2337: positiveSegments: 5, negativeSegments: 6 2339: positiveSegments: 11, negativeSegments: 0 2340: positiveSegments: 25, negativeSegments: 0 2341: positiveSegments: 3, negativeSegments: 3 2345: positiveSegments: 4, negativeSegments: 3 2346: positiveSegments: 0, negativeSegments: 3 2348: positiveSegments: 4, negativeSegments: 6 2349: positiveSegments: 16, negativeSegments: 6 2352: positiveSegments: 13, negativeSegments: 15 2353: positiveSegments: 9, negativeSegments: 15 2354: positiveSegments: 3, negativeSegments: 12 2356: positiveSegments: 14, negativeSegments: 0 2357: positiveSegments: 0, negativeSegments: 6 2359: positiveSegments: 18, negativeSegments: 0 2364: positiveSegments: 6, negativeSegments: 0 2365: positiveSegments: 4, negativeSegments: 0 2371: positiveSegments: 14, negativeSegments: 0 2372: positiveSegments: 0, negativeSegments: 6 2373: positiveSegments: 17, negativeSegments: 6 2375: positiveSegments: 16, negativeSegments: 15 2377: positiveSegments: 10, negativeSegments: 3 2379: positiveSegments: 4, negativeSegments: 0 2380: positiveSegments: 8, negativeSegments: 12 2382: positiveSegments: 5, negativeSegments: 0 2383: positiveSegments: 19, negativeSegments: 3 2389: positiveSegments: 15, negativeSegments: 9 2392: positiveSegments: 4, negativeSegments: 9 2393: positiveSegments: 12, negativeSegments: 3 2394: positiveSegments: 0, negativeSegments: 6 2396: positiveSegments: 17, negativeSegments: 0 2401: positiveSegments: 0, negativeSegments: 6 2405: positiveSegments: 16, negativeSegments: 3 2409: positiveSegments: 4, negativeSegments: 0 2411: positiveSegments: 14, negativeSegments: 0 2412: exit early, no segments to save 2412: positiveSegments: 0, negativeSegments: 0 2413: positiveSegments: 4, negativeSegments: 0 2416: positiveSegments: 7, negativeSegments: 9 2417: positiveSegments: 4, negativeSegments: 3 2419: positiveSegments: 4, negativeSegments: 0 2420: positiveSegments: 0, negativeSegments: 21 2421: positiveSegments: 0, negativeSegments: 9 2422: positiveSegments: 11, negativeSegments: 0 2424: positiveSegments: 23, negativeSegments: 3 2425: positiveSegments: 4, negativeSegments: 6 2427: positiveSegments: 2, negativeSegments: 9 2428: positiveSegments: 7, negativeSegments: 3 2432: positiveSegments: 11, negativeSegments: 0 2433: positiveSegments: 38, negativeSegments: 21 2434: exit early, no segments to save 2434: positiveSegments: 0, negativeSegments: 0 2435: positiveSegments: 4, negativeSegments: 3 2436: positiveSegments: 0, negativeSegments: 3 2438: positiveSegments: 0, negativeSegments: 6 2441: positiveSegments: 23, negativeSegments: 6 2442: positiveSegments: 6, negativeSegments: 0 2443: positiveSegments: 7, negativeSegments: 30 2444: positiveSegments: 6, negativeSegments: 0 2445: positiveSegments: 12, negativeSegments: 3 2447: positiveSegments: 0, negativeSegments: 3 2450: exit early, no segments to save 2450: positiveSegments: 0, negativeSegments: 0 2452: positiveSegments: 17, negativeSegments: 0 2453: positiveSegments: 30, negativeSegments: 6 2455: positiveSegments: 6, negativeSegments: 6 2458: positiveSegments: 25, negativeSegments: 9 2460: positiveSegments: 3, negativeSegments: 0 2462: positiveSegments: 17, negativeSegments: 3 2466: positiveSegments: 2, negativeSegments: 0 count processed: 1200, current case index: 2469 2469: positiveSegments: 4, negativeSegments: 6 2470: positiveSegments: 24, negativeSegments: 0 2471: positiveSegments: 10, negativeSegments: 0 2472: positiveSegments: 8, negativeSegments: 6 2473: exit early, no segments to save 2473: positiveSegments: 0, negativeSegments: 0 2474: positiveSegments: 2, negativeSegments: 6 2479: exit early, no segments to save 2479: positiveSegments: 0, negativeSegments: 0 2480: positiveSegments: 4, negativeSegments: 9 2481: positiveSegments: 0, negativeSegments: 3 2482: positiveSegments: 0, negativeSegments: 3 2483: positiveSegments: 4, negativeSegments: 0 2485: positiveSegments: 0, negativeSegments: 6 2487: positiveSegments: 35, negativeSegments: 9 2489: positiveSegments: 38, negativeSegments: 6 2493: positiveSegments: 26, negativeSegments: 3 2494: positiveSegments: 19, negativeSegments: 0 2495: positiveSegments: 4, negativeSegments: 9 2496: exit early, no segments to save 2496: positiveSegments: 0, negativeSegments: 0 2497: positiveSegments: 12, negativeSegments: 9 2500: positiveSegments: 4, negativeSegments: 0 2501: positiveSegments: 0, negativeSegments: 18 2503: positiveSegments: 0, negativeSegments: 3 2507: positiveSegments: 6, negativeSegments: 6 2508: positiveSegments: 0, negativeSegments: 9 2509: exit early, no segments to save 2509: positiveSegments: 0, negativeSegments: 0 2510: positiveSegments: 7, negativeSegments: 0 2511: positiveSegments: 5, negativeSegments: 0 2516: exit early, no segments to save 2516: positiveSegments: 0, negativeSegments: 0 2517: positiveSegments: 10, negativeSegments: 0 2519: positiveSegments: 25, negativeSegments: 3 2521: positiveSegments: 14, negativeSegments: 0 2523: positiveSegments: 8, negativeSegments: 3 2527: positiveSegments: 14, negativeSegments: 6 2528: exit early, no segments to save 2528: positiveSegments: 0, negativeSegments: 0 2532: positiveSegments: 4, negativeSegments: 0 2533: positiveSegments: 5, negativeSegments: 0 2535: positiveSegments: 4, negativeSegments: 0 2537: positiveSegments: 8, negativeSegments: 6 2539: positiveSegments: 3, negativeSegments: 0 2542: positiveSegments: 0, negativeSegments: 3 2544: positiveSegments: 10, negativeSegments: 12 2547: positiveSegments: 16, negativeSegments: 6 2549: exit early, no segments to save 2549: positiveSegments: 0, negativeSegments: 0 2553: positiveSegments: 27, negativeSegments: 0 2558: positiveSegments: 4, negativeSegments: 3 2559: positiveSegments: 14, negativeSegments: 0 2561: exit early, no segments to save 2561: positiveSegments: 0, negativeSegments: 0 2562: positiveSegments: 0, negativeSegments: 6 2566: positiveSegments: 0, negativeSegments: 3 2568: positiveSegments: 1, negativeSegments: 12 2569: exit early, no segments to save 2569: positiveSegments: 0, negativeSegments: 0 2572: exit early, no segments to save 2572: positiveSegments: 0, negativeSegments: 0 2575: exit early, no segments to save 2575: positiveSegments: 0, negativeSegments: 0 2576: exit early, no segments to save 2576: positiveSegments: 0, negativeSegments: 0 2578: positiveSegments: 4, negativeSegments: 6 2580: positiveSegments: 21, negativeSegments: 6 2583: positiveSegments: 6, negativeSegments: 0 2584: positiveSegments: 17, negativeSegments: 9 2585: positiveSegments: 18, negativeSegments: 0 2587: positiveSegments: 14, negativeSegments: 0 2589: positiveSegments: 0, negativeSegments: 6 2590: positiveSegments: 1, negativeSegments: 6 2593: exit early, no segments to save 2593: positiveSegments: 0, negativeSegments: 0 2594: positiveSegments: 10, negativeSegments: 3 2596: positiveSegments: 6, negativeSegments: 3 2597: positiveSegments: 2, negativeSegments: 0 2601: positiveSegments: 6, negativeSegments: 6 2605: positiveSegments: 1, negativeSegments: 0 2606: positiveSegments: 8, negativeSegments: 9 2607: exit early, no segments to save 2607: positiveSegments: 0, negativeSegments: 0 2608: positiveSegments: 10, negativeSegments: 0 2609: positiveSegments: 4, negativeSegments: 3 2611: positiveSegments: 18, negativeSegments: 3 2612: positiveSegments: 10, negativeSegments: 0 2613: positiveSegments: 0, negativeSegments: 3 2616: exit early, no segments to save 2616: positiveSegments: 0, negativeSegments: 0 2618: positiveSegments: 7, negativeSegments: 15 2619: positiveSegments: 11, negativeSegments: 3 2622: positiveSegments: 17, negativeSegments: 3 2624: positiveSegments: 8, negativeSegments: 3 2627: positiveSegments: 8, negativeSegments: 0 2628: positiveSegments: 2, negativeSegments: 9 2630: positiveSegments: 6, negativeSegments: 6 2631: positiveSegments: 12, negativeSegments: 0 2632: positiveSegments: 15, negativeSegments: 0 2637: positiveSegments: 2, negativeSegments: 3 2639: positiveSegments: 4, negativeSegments: 0 2641: positiveSegments: 0, negativeSegments: 6 2644: positiveSegments: 9, negativeSegments: 0 2646: exit early, no segments to save 2646: positiveSegments: 0, negativeSegments: 0 2648: positiveSegments: 12, negativeSegments: 9 2652: positiveSegments: 0, negativeSegments: 15 2654: positiveSegments: 0, negativeSegments: 3 2655: positiveSegments: 3, negativeSegments: 6 2656: positiveSegments: 0, negativeSegments: 12 2657: positiveSegments: 0, negativeSegments: 9 2658: positiveSegments: 13, negativeSegments: 6 2662: positiveSegments: 11, negativeSegments: 21 2663: positiveSegments: 4, negativeSegments: 6 2664: positiveSegments: 8, negativeSegments: 15 count processed: 1300, current case index: 2665 2665: positiveSegments: 0, negativeSegments: 3 2667: positiveSegments: 17, negativeSegments: 9 2670: exit early, no segments to save 2670: positiveSegments: 0, negativeSegments: 0 2671: positiveSegments: 13, negativeSegments: 3 2673: exit early, no segments to save 2673: positiveSegments: 0, negativeSegments: 0 2674: positiveSegments: 8, negativeSegments: 9 2676: positiveSegments: 4, negativeSegments: 0 2678: positiveSegments: 4, negativeSegments: 3 2680: positiveSegments: 17, negativeSegments: 0 2687: positiveSegments: 9, negativeSegments: 0 2688: positiveSegments: 23, negativeSegments: 3 2690: positiveSegments: 8, negativeSegments: 3 2693: positiveSegments: 19, negativeSegments: 6 2695: exit early, no segments to save 2695: positiveSegments: 0, negativeSegments: 0 2697: positiveSegments: 0, negativeSegments: 9 2698: positiveSegments: 4, negativeSegments: 0 2699: positiveSegments: 5, negativeSegments: 9 2700: positiveSegments: 7, negativeSegments: 0 2701: positiveSegments: 0, negativeSegments: 6 2703: positiveSegments: 4, negativeSegments: 15 2705: positiveSegments: 2, negativeSegments: 9 2706: positiveSegments: 36, negativeSegments: 3 2712: positiveSegments: 13, negativeSegments: 3 2713: positiveSegments: 0, negativeSegments: 3 2716: positiveSegments: 0, negativeSegments: 6 2717: positiveSegments: 8, negativeSegments: 0 2722: positiveSegments: 50, negativeSegments: 0 2724: positiveSegments: 10, negativeSegments: 12 2732: exit early, no segments to save 2732: positiveSegments: 0, negativeSegments: 0 2735: positiveSegments: 4, negativeSegments: 3 2736: exit early, no segments to save 2736: positiveSegments: 0, negativeSegments: 0 2738: positiveSegments: 9, negativeSegments: 3 2741: positiveSegments: 18, negativeSegments: 6 2742: positiveSegments: 19, negativeSegments: 3 2744: positiveSegments: 4, negativeSegments: 6 2746: positiveSegments: 4, negativeSegments: 0 2747: positiveSegments: 7, negativeSegments: 9 2749: positiveSegments: 10, negativeSegments: 3 2750: positiveSegments: 4, negativeSegments: 6 2751: positiveSegments: 25, negativeSegments: 6 2755: positiveSegments: 12, negativeSegments: 9 2760: exit early, no segments to save 2760: positiveSegments: 0, negativeSegments: 0 2761: positiveSegments: 10, negativeSegments: 3 2762: positiveSegments: 0, negativeSegments: 3 2763: positiveSegments: 4, negativeSegments: 0 2764: positiveSegments: 26, negativeSegments: 6 2765: positiveSegments: 14, negativeSegments: 0 2766: positiveSegments: 0, negativeSegments: 12 2769: positiveSegments: 4, negativeSegments: 9 2771: exit early, no segments to save 2771: positiveSegments: 0, negativeSegments: 0 2774: positiveSegments: 5, negativeSegments: 0 2775: positiveSegments: 18, negativeSegments: 3 2777: positiveSegments: 4, negativeSegments: 3 2778: positiveSegments: 16, negativeSegments: 6 2779: positiveSegments: 4, negativeSegments: 0 2781: positiveSegments: 21, negativeSegments: 9 2783: positiveSegments: 18, negativeSegments: 0 2785: positiveSegments: 25, negativeSegments: 3 2788: positiveSegments: 3, negativeSegments: 0 2795: positiveSegments: 9, negativeSegments: 6 2799: positiveSegments: 4, negativeSegments: 0 2800: positiveSegments: 5, negativeSegments: 12 2803: positiveSegments: 2, negativeSegments: 0 2804: positiveSegments: 12, negativeSegments: 9 2806: positiveSegments: 3, negativeSegments: 0 2807: positiveSegments: 5, negativeSegments: 6 2809: positiveSegments: 11, negativeSegments: 3 2814: positiveSegments: 10, negativeSegments: 6 2815: positiveSegments: 19, negativeSegments: 0 2819: positiveSegments: 17, negativeSegments: 0 2820: positiveSegments: 5, negativeSegments: 9 2823: positiveSegments: 13, negativeSegments: 3 2824: positiveSegments: 22, negativeSegments: 9 2827: positiveSegments: 17, negativeSegments: 0 2828: positiveSegments: 26, negativeSegments: 6 2829: positiveSegments: 15, negativeSegments: 3 2830: positiveSegments: 8, negativeSegments: 0 2835: positiveSegments: 30, negativeSegments: 0 2836: positiveSegments: 0, negativeSegments: 9 2837: positiveSegments: 11, negativeSegments: 6 2839: positiveSegments: 5, negativeSegments: 3 2845: exit early, no segments to save 2845: positiveSegments: 0, negativeSegments: 0 2847: positiveSegments: 4, negativeSegments: 12 2848: positiveSegments: 6, negativeSegments: 12 2849: positiveSegments: 15, negativeSegments: 0 2850: positiveSegments: 6, negativeSegments: 6 2851: positiveSegments: 23, negativeSegments: 3 2854: positiveSegments: 4, negativeSegments: 3 2858: positiveSegments: 25, negativeSegments: 3 2859: positiveSegments: 5, negativeSegments: 0 2860: positiveSegments: 8, negativeSegments: 18 2861: positiveSegments: 10, negativeSegments: 0 2863: positiveSegments: 4, negativeSegments: 6 2864: positiveSegments: 8, negativeSegments: 3 2868: positiveSegments: 6, negativeSegments: 0 2871: positiveSegments: 3, negativeSegments: 3 2872: positiveSegments: 20, negativeSegments: 6 2876: positiveSegments: 13, negativeSegments: 9 2877: nothing saved, all segments filtered 2877: positiveSegments: 2, negativeSegments: 0 2883: positiveSegments: 8, negativeSegments: 3 count processed: 1400, current case index: 2888 2888: positiveSegments: 2, negativeSegments: 9 2889: positiveSegments: 12, negativeSegments: 0 2890: positiveSegments: 6, negativeSegments: 0 2891: positiveSegments: 11, negativeSegments: 0 2895: exit early, no segments to save 2895: positiveSegments: 0, negativeSegments: 0 2900: positiveSegments: 5, negativeSegments: 0 2903: positiveSegments: 6, negativeSegments: 0 2905: positiveSegments: 8, negativeSegments: 9 2906: exit early, no segments to save 2906: positiveSegments: 0, negativeSegments: 0 2909: exit early, no segments to save 2909: positiveSegments: 0, negativeSegments: 0 2910: positiveSegments: 0, negativeSegments: 3 2911: positiveSegments: 25, negativeSegments: 12 2914: positiveSegments: 0, negativeSegments: 6 2919: positiveSegments: 4, negativeSegments: 0 2922: positiveSegments: 4, negativeSegments: 0 2924: positiveSegments: 12, negativeSegments: 0 2927: positiveSegments: 0, negativeSegments: 3 2929: positiveSegments: 10, negativeSegments: 0 2930: positiveSegments: 1, negativeSegments: 6 2931: positiveSegments: 4, negativeSegments: 3 2935: positiveSegments: 17, negativeSegments: 0 2939: positiveSegments: 0, negativeSegments: 12 2940: positiveSegments: 18, negativeSegments: 6 2943: positiveSegments: 4, negativeSegments: 3 2944: positiveSegments: 4, negativeSegments: 9 2945: positiveSegments: 34, negativeSegments: 9 2947: positiveSegments: 20, negativeSegments: 0 2949: positiveSegments: 7, negativeSegments: 0 2952: positiveSegments: 4, negativeSegments: 21 2954: positiveSegments: 0, negativeSegments: 15 2955: positiveSegments: 4, negativeSegments: 3 2956: positiveSegments: 0, negativeSegments: 3 2958: positiveSegments: 0, negativeSegments: 3 2959: positiveSegments: 9, negativeSegments: 12 2960: positiveSegments: 25, negativeSegments: 0 2961: positiveSegments: 22, negativeSegments: 9 2964: positiveSegments: 13, negativeSegments: 0 2966: positiveSegments: 7, negativeSegments: 3 2970: positiveSegments: 3, negativeSegments: 12 2971: positiveSegments: 0, negativeSegments: 12 2972: positiveSegments: 9, negativeSegments: 0 2974: positiveSegments: 4, negativeSegments: 6 2975: positiveSegments: 4, negativeSegments: 9 2977: positiveSegments: 9, negativeSegments: 0 2980: positiveSegments: 8, negativeSegments: 3 2981: positiveSegments: 0, negativeSegments: 9 2982: positiveSegments: 13, negativeSegments: 9 2987: positiveSegments: 0, negativeSegments: 3 2991: positiveSegments: 17, negativeSegments: 6 2992: positiveSegments: 19, negativeSegments: 3 2993: positiveSegments: 17, negativeSegments: 3 2998: positiveSegments: 0, negativeSegments: 3 2999: positiveSegments: 8, negativeSegments: 0 3000: positiveSegments: 14, negativeSegments: 0 3001: positiveSegments: 11, negativeSegments: 3 3003: positiveSegments: 8, negativeSegments: 0 3004: positiveSegments: 0, negativeSegments: 9 3006: positiveSegments: 17, negativeSegments: 0 3009: positiveSegments: 26, negativeSegments: 3 3014: positiveSegments: 12, negativeSegments: 3 3015: nothing saved, all segments filtered 3015: positiveSegments: 1, negativeSegments: 0 3019: positiveSegments: 29, negativeSegments: 0 3020: positiveSegments: 5, negativeSegments: 0 3023: positiveSegments: 10, negativeSegments: 3 3024: positiveSegments: 29, negativeSegments: 3 3027: positiveSegments: 4, negativeSegments: 0 3028: positiveSegments: 16, negativeSegments: 3 3030: positiveSegments: 6, negativeSegments: 6 3034: positiveSegments: 13, negativeSegments: 0 3035: positiveSegments: 8, negativeSegments: 6 3037: positiveSegments: 21, negativeSegments: 12 3039: positiveSegments: 13, negativeSegments: 0 3040: positiveSegments: 23, negativeSegments: 6 3042: positiveSegments: 8, negativeSegments: 3 3044: positiveSegments: 20, negativeSegments: 3 3045: positiveSegments: 0, negativeSegments: 3 3046: exit early, no segments to save 3046: positiveSegments: 0, negativeSegments: 0 3047: positiveSegments: 12, negativeSegments: 0 3048: positiveSegments: 0, negativeSegments: 9 3050: positiveSegments: 6, negativeSegments: 0 3051: positiveSegments: 35, negativeSegments: 9 3057: positiveSegments: 5, negativeSegments: 3 3058: positiveSegments: 36, negativeSegments: 0 3059: positiveSegments: 4, negativeSegments: 9 3060: positiveSegments: 11, negativeSegments: 6 3065: positiveSegments: 5, negativeSegments: 0 3070: positiveSegments: 0, negativeSegments: 3 3072: exit early, no segments to save 3072: positiveSegments: 0, negativeSegments: 0 3073: positiveSegments: 4, negativeSegments: 3 3074: positiveSegments: 32, negativeSegments: 6 3075: positiveSegments: 8, negativeSegments: 3 3078: exit early, no segments to save 3078: positiveSegments: 0, negativeSegments: 0 3079: positiveSegments: 12, negativeSegments: 6 3080: positiveSegments: 2, negativeSegments: 0 3082: positiveSegments: 10, negativeSegments: 6 3085: positiveSegments: 6, negativeSegments: 0 3087: positiveSegments: 14, negativeSegments: 0 3088: positiveSegments: 5, negativeSegments: 0 3090: positiveSegments: 5, negativeSegments: 6 3091: positiveSegments: 8, negativeSegments: 0 count processed: 1500, current case index: 3092 3092: positiveSegments: 2, negativeSegments: 6 3093: positiveSegments: 14, negativeSegments: 12 3094: positiveSegments: 8, negativeSegments: 6 3095: positiveSegments: 12, negativeSegments: 0 3096: positiveSegments: 15, negativeSegments: 3 3097: positiveSegments: 12, negativeSegments: 0 3098: positiveSegments: 4, negativeSegments: 9 3101: positiveSegments: 15, negativeSegments: 0 3102: positiveSegments: 1, negativeSegments: 3 3103: positiveSegments: 0, negativeSegments: 3 3106: positiveSegments: 0, negativeSegments: 6 3107: positiveSegments: 12, negativeSegments: 3 3108: positiveSegments: 15, negativeSegments: 9 3110: positiveSegments: 0, negativeSegments: 9 3112: positiveSegments: 4, negativeSegments: 3 3113: positiveSegments: 24, negativeSegments: 6 3114: exit early, no segments to save 3114: positiveSegments: 0, negativeSegments: 0 3116: positiveSegments: 8, negativeSegments: 0 3118: positiveSegments: 0, negativeSegments: 12 3120: positiveSegments: 11, negativeSegments: 0 3121: positiveSegments: 0, negativeSegments: 12 3122: positiveSegments: 5, negativeSegments: 3 3125: positiveSegments: 0, negativeSegments: 6 3126: positiveSegments: 8, negativeSegments: 0 3128: exit early, no segments to save 3128: positiveSegments: 0, negativeSegments: 0 3130: positiveSegments: 0, negativeSegments: 9 3133: positiveSegments: 24, negativeSegments: 3 3134: positiveSegments: 15, negativeSegments: 21 3135: positiveSegments: 0, negativeSegments: 3 3136: positiveSegments: 12, negativeSegments: 0 3137: positiveSegments: 4, negativeSegments: 12 3138: positiveSegments: 18, negativeSegments: 0 3141: positiveSegments: 4, negativeSegments: 3 3142: positiveSegments: 15, negativeSegments: 0 3145: positiveSegments: 20, negativeSegments: 9 3146: positiveSegments: 32, negativeSegments: 3 3147: positiveSegments: 13, negativeSegments: 6 3148: positiveSegments: 29, negativeSegments: 12 3149: positiveSegments: 4, negativeSegments: 12 3150: positiveSegments: 4, negativeSegments: 9 3153: positiveSegments: 7, negativeSegments: 0 3154: positiveSegments: 22, negativeSegments: 6 3155: positiveSegments: 12, negativeSegments: 9 3157: positiveSegments: 16, negativeSegments: 0 3159: positiveSegments: 8, negativeSegments: 6 3161: positiveSegments: 16, negativeSegments: 3 3164: positiveSegments: 4, negativeSegments: 9 3165: positiveSegments: 26, negativeSegments: 0 3167: positiveSegments: 8, negativeSegments: 12 3169: positiveSegments: 12, negativeSegments: 0 3170: exit early, no segments to save 3170: positiveSegments: 0, negativeSegments: 0 3172: positiveSegments: 0, negativeSegments: 3 3173: positiveSegments: 23, negativeSegments: 3 3175: positiveSegments: 0, negativeSegments: 6 3176: exit early, no segments to save 3176: positiveSegments: 0, negativeSegments: 0 3180: positiveSegments: 8, negativeSegments: 0 3181: positiveSegments: 24, negativeSegments: 12 3184: positiveSegments: 4, negativeSegments: 6 3186: positiveSegments: 10, negativeSegments: 9 3188: positiveSegments: 33, negativeSegments: 0 3189: positiveSegments: 0, negativeSegments: 9 3193: positiveSegments: 20, negativeSegments: 24 3194: positiveSegments: 0, negativeSegments: 18 3196: positiveSegments: 0, negativeSegments: 9 3198: positiveSegments: 5, negativeSegments: 0 3200: positiveSegments: 4, negativeSegments: 0 3202: positiveSegments: 16, negativeSegments: 0 3203: positiveSegments: 4, negativeSegments: 12 3205: positiveSegments: 2, negativeSegments: 0 3207: exit early, no segments to save 3207: positiveSegments: 0, negativeSegments: 0 3208: positiveSegments: 0, negativeSegments: 3 3211: positiveSegments: 0, negativeSegments: 6 3216: positiveSegments: 9, negativeSegments: 3 3218: positiveSegments: 4, negativeSegments: 3 3221: positiveSegments: 3, negativeSegments: 3 3222: positiveSegments: 26, negativeSegments: 0 3223: positiveSegments: 0, negativeSegments: 12 3226: positiveSegments: 7, negativeSegments: 0 3228: positiveSegments: 35, negativeSegments: 6 3229: positiveSegments: 0, negativeSegments: 9 3231: positiveSegments: 12, negativeSegments: 6 3232: positiveSegments: 6, negativeSegments: 9 3233: positiveSegments: 22, negativeSegments: 3 3235: positiveSegments: 9, negativeSegments: 0 3240: positiveSegments: 4, negativeSegments: 6 3243: positiveSegments: 0, negativeSegments: 6 3247: positiveSegments: 33, negativeSegments: 6 3248: positiveSegments: 9, negativeSegments: 0 3249: positiveSegments: 2, negativeSegments: 0 3250: positiveSegments: 12, negativeSegments: 6 3255: positiveSegments: 27, negativeSegments: 0 3256: positiveSegments: 4, negativeSegments: 6 3260: positiveSegments: 6, negativeSegments: 3 3263: positiveSegments: 4, negativeSegments: 6 3267: positiveSegments: 0, negativeSegments: 18 3270: positiveSegments: 48, negativeSegments: 0 3271: positiveSegments: 15, negativeSegments: 6 3273: positiveSegments: 40, negativeSegments: 0 3275: positiveSegments: 9, negativeSegments: 3 3276: positiveSegments: 10, negativeSegments: 0 count processed: 1600, current case index: 3279 3279: positiveSegments: 23, negativeSegments: 3 3284: positiveSegments: 20, negativeSegments: 6 3286: positiveSegments: 21, negativeSegments: 0 3287: positiveSegments: 12, negativeSegments: 12 3291: positiveSegments: 26, negativeSegments: 6 3293: positiveSegments: 30, negativeSegments: 6 3295: positiveSegments: 8, negativeSegments: 0 3296: positiveSegments: 0, negativeSegments: 3 3297: positiveSegments: 18, negativeSegments: 3 3299: exit early, no segments to save 3299: positiveSegments: 0, negativeSegments: 0 3304: positiveSegments: 8, negativeSegments: 3 3307: positiveSegments: 13, negativeSegments: 6 3309: positiveSegments: 8, negativeSegments: 6 3310: positiveSegments: 26, negativeSegments: 0 3311: positiveSegments: 26, negativeSegments: 12 3312: positiveSegments: 6, negativeSegments: 3 3315: positiveSegments: 0, negativeSegments: 9 3317: positiveSegments: 8, negativeSegments: 6 3318: positiveSegments: 12, negativeSegments: 9 3321: positiveSegments: 4, negativeSegments: 6 3325: positiveSegments: 12, negativeSegments: 6 3327: positiveSegments: 4, negativeSegments: 3 3328: positiveSegments: 17, negativeSegments: 3 3329: positiveSegments: 9, negativeSegments: 0 3330: positiveSegments: 7, negativeSegments: 12 3331: positiveSegments: 19, negativeSegments: 3 3333: positiveSegments: 8, negativeSegments: 3 3334: positiveSegments: 33, negativeSegments: 9 3336: positiveSegments: 12, negativeSegments: 0 3338: positiveSegments: 14, negativeSegments: 3 3340: positiveSegments: 14, negativeSegments: 0 3342: positiveSegments: 8, negativeSegments: 9 3344: positiveSegments: 4, negativeSegments: 3 3345: positiveSegments: 24, negativeSegments: 3 3348: positiveSegments: 21, negativeSegments: 9 3352: positiveSegments: 18, negativeSegments: 6 3355: positiveSegments: 4, negativeSegments: 0 3357: positiveSegments: 4, negativeSegments: 0 3358: positiveSegments: 9, negativeSegments: 12 3359: positiveSegments: 10, negativeSegments: 0 3361: positiveSegments: 1, negativeSegments: 21 3362: exit early, no segments to save 3362: positiveSegments: 0, negativeSegments: 0 3363: positiveSegments: 4, negativeSegments: 12 3364: positiveSegments: 17, negativeSegments: 6 3365: exit early, no segments to save 3365: positiveSegments: 0, negativeSegments: 0 3366: exit early, no segments to save 3366: positiveSegments: 0, negativeSegments: 0 3367: positiveSegments: 3, negativeSegments: 3 3369: positiveSegments: 0, negativeSegments: 3 3373: positiveSegments: 30, negativeSegments: 9 3374: positiveSegments: 3, negativeSegments: 3 3375: positiveSegments: 1, negativeSegments: 3 3376: positiveSegments: 4, negativeSegments: 0 3377: exit early, no segments to save 3377: positiveSegments: 0, negativeSegments: 0 3379: positiveSegments: 0, negativeSegments: 3 3380: positiveSegments: 16, negativeSegments: 9 3381: positiveSegments: 0, negativeSegments: 15 3382: positiveSegments: 16, negativeSegments: 3 3383: positiveSegments: 8, negativeSegments: 3 3385: positiveSegments: 4, negativeSegments: 9 3389: positiveSegments: 5, negativeSegments: 3 3391: exit early, no segments to save 3391: positiveSegments: 0, negativeSegments: 0 3394: positiveSegments: 12, negativeSegments: 12 3396: positiveSegments: 3, negativeSegments: 0 3398: positiveSegments: 6, negativeSegments: 3 3399: positiveSegments: 4, negativeSegments: 3 3401: positiveSegments: 12, negativeSegments: 3 3404: positiveSegments: 18, negativeSegments: 3 3407: positiveSegments: 0, negativeSegments: 3 3409: positiveSegments: 14, negativeSegments: 3 3412: positiveSegments: 3, negativeSegments: 0 3413: exit early, no segments to save 3413: positiveSegments: 0, negativeSegments: 0 3414: exit early, no segments to save 3414: positiveSegments: 0, negativeSegments: 0 3415: positiveSegments: 28, negativeSegments: 6 3418: positiveSegments: 4, negativeSegments: 6 3422: positiveSegments: 4, negativeSegments: 3 3426: positiveSegments: 6, negativeSegments: 0 3427: positiveSegments: 4, negativeSegments: 0 3428: positiveSegments: 18, negativeSegments: 3 3429: positiveSegments: 15, negativeSegments: 6 3431: positiveSegments: 0, negativeSegments: 6 3434: positiveSegments: 0, negativeSegments: 3 3435: positiveSegments: 0, negativeSegments: 3 3436: positiveSegments: 4, negativeSegments: 6 3440: positiveSegments: 2, negativeSegments: 6 3441: positiveSegments: 0, negativeSegments: 3 3442: positiveSegments: 20, negativeSegments: 3 3447: positiveSegments: 11, negativeSegments: 3 3449: positiveSegments: 12, negativeSegments: 3 3450: positiveSegments: 8, negativeSegments: 6 3451: positiveSegments: 8, negativeSegments: 9 3453: positiveSegments: 0, negativeSegments: 6 3457: positiveSegments: 14, negativeSegments: 6 3458: positiveSegments: 8, negativeSegments: 3 3460: positiveSegments: 10, negativeSegments: 0 3462: positiveSegments: 9, negativeSegments: 6 3463: positiveSegments: 4, negativeSegments: 9 3464: positiveSegments: 8, negativeSegments: 6 3468: positiveSegments: 4, negativeSegments: 12 3470: positiveSegments: 12, negativeSegments: 0 3472: positiveSegments: 6, negativeSegments: 0 count processed: 1700, current case index: 3475 3475: positiveSegments: 4, negativeSegments: 3 3476: exit early, no segments to save 3476: positiveSegments: 0, negativeSegments: 0 3478: positiveSegments: 12, negativeSegments: 3 3479: positiveSegments: 0, negativeSegments: 3 3481: positiveSegments: 16, negativeSegments: 0 3483: positiveSegments: 24, negativeSegments: 15 3488: positiveSegments: 4, negativeSegments: 6 3492: positiveSegments: 10, negativeSegments: 0 3499: positiveSegments: 0, negativeSegments: 15 3500: exit early, no segments to save 3500: positiveSegments: 0, negativeSegments: 0 3501: positiveSegments: 4, negativeSegments: 3 3502: positiveSegments: 8, negativeSegments: 9 3503: positiveSegments: 5, negativeSegments: 9 3505: positiveSegments: 4, negativeSegments: 6 3506: positiveSegments: 17, negativeSegments: 0 3513: positiveSegments: 0, negativeSegments: 15 3514: positiveSegments: 12, negativeSegments: 0 3515: positiveSegments: 8, negativeSegments: 6 3516: positiveSegments: 12, negativeSegments: 9 3517: positiveSegments: 7, negativeSegments: 0 3518: exit early, no segments to save 3518: positiveSegments: 0, negativeSegments: 0 3521: positiveSegments: 38, negativeSegments: 6 3524: positiveSegments: 37, negativeSegments: 3 3526: positiveSegments: 3, negativeSegments: 3 3527: positiveSegments: 4, negativeSegments: 6 3528: positiveSegments: 0, negativeSegments: 3 3529: positiveSegments: 6, negativeSegments: 0 3532: positiveSegments: 2, negativeSegments: 3 3533: exit early, no segments to save 3533: positiveSegments: 0, negativeSegments: 0 3535: positiveSegments: 14, negativeSegments: 0 3537: positiveSegments: 4, negativeSegments: 9 3538: exit early, no segments to save 3538: positiveSegments: 0, negativeSegments: 0 3544: positiveSegments: 0, negativeSegments: 6 3545: positiveSegments: 7, negativeSegments: 0 3546: positiveSegments: 23, negativeSegments: 6 3549: positiveSegments: 27, negativeSegments: 0 3550: positiveSegments: 22, negativeSegments: 3 3555: positiveSegments: 0, negativeSegments: 6 3558: positiveSegments: 4, negativeSegments: 12 3559: positiveSegments: 25, negativeSegments: 9 3560: positiveSegments: 3, negativeSegments: 6 3562: positiveSegments: 1, negativeSegments: 3 3564: positiveSegments: 2, negativeSegments: 3 3565: positiveSegments: 12, negativeSegments: 6 3566: positiveSegments: 30, negativeSegments: 3 3567: positiveSegments: 0, negativeSegments: 6 3568: positiveSegments: 11, negativeSegments: 3 3569: positiveSegments: 10, negativeSegments: 0 3570: positiveSegments: 4, negativeSegments: 0 3571: positiveSegments: 8, negativeSegments: 15 3572: positiveSegments: 36, negativeSegments: 6 3573: positiveSegments: 8, negativeSegments: 3 3576: positiveSegments: 9, negativeSegments: 6 3581: positiveSegments: 4, negativeSegments: 3 3582: positiveSegments: 11, negativeSegments: 9 3585: positiveSegments: 9, negativeSegments: 0 3588: positiveSegments: 34, negativeSegments: 0 3589: positiveSegments: 0, negativeSegments: 3 3593: positiveSegments: 4, negativeSegments: 9 3594: positiveSegments: 4, negativeSegments: 18 3596: positiveSegments: 0, negativeSegments: 3 3602: positiveSegments: 7, negativeSegments: 6 3603: positiveSegments: 8, negativeSegments: 3 3606: positiveSegments: 0, negativeSegments: 12 3607: positiveSegments: 32, negativeSegments: 0 3608: positiveSegments: 4, negativeSegments: 3 3609: positiveSegments: 14, negativeSegments: 0 3611: positiveSegments: 2, negativeSegments: 3 3614: positiveSegments: 8, negativeSegments: 0 3616: positiveSegments: 27, negativeSegments: 0 3618: positiveSegments: 0, negativeSegments: 9 3620: positiveSegments: 6, negativeSegments: 3 3621: positiveSegments: 22, negativeSegments: 6 3623: positiveSegments: 17, negativeSegments: 0 3625: positiveSegments: 58, negativeSegments: 15 3628: positiveSegments: 7, negativeSegments: 3 3629: positiveSegments: 0, negativeSegments: 3 3631: positiveSegments: 25, negativeSegments: 0 3642: positiveSegments: 0, negativeSegments: 12 3648: positiveSegments: 5, negativeSegments: 0 3652: positiveSegments: 4, negativeSegments: 18 3656: positiveSegments: 4, negativeSegments: 21 3658: positiveSegments: 0, negativeSegments: 9 3660: positiveSegments: 13, negativeSegments: 0 3662: positiveSegments: 2, negativeSegments: 9 3663: positiveSegments: 2, negativeSegments: 9 3668: positiveSegments: 0, negativeSegments: 15 3669: positiveSegments: 14, negativeSegments: 0 3672: positiveSegments: 7, negativeSegments: 0 3674: positiveSegments: 0, negativeSegments: 12 3677: positiveSegments: 4, negativeSegments: 6 3678: positiveSegments: 4, negativeSegments: 3 3680: exit early, no segments to save 3680: positiveSegments: 0, negativeSegments: 0 3682: positiveSegments: 13, negativeSegments: 0 3686: positiveSegments: 21, negativeSegments: 6 3687: positiveSegments: 7, negativeSegments: 6 3688: positiveSegments: 2, negativeSegments: 0 3689: positiveSegments: 16, negativeSegments: 3 3690: positiveSegments: 4, negativeSegments: 3 3691: exit early, no segments to save 3691: positiveSegments: 0, negativeSegments: 0 count processed: 1800, current case index: 3694 3694: positiveSegments: 16, negativeSegments: 9 3697: positiveSegments: 0, negativeSegments: 9 3699: positiveSegments: 11, negativeSegments: 6 3700: positiveSegments: 16, negativeSegments: 3 3702: positiveSegments: 8, negativeSegments: 6 3703: positiveSegments: 4, negativeSegments: 6 3704: positiveSegments: 4, negativeSegments: 6 3706: positiveSegments: 3, negativeSegments: 18 3709: positiveSegments: 2, negativeSegments: 0 3710: positiveSegments: 4, negativeSegments: 6 3711: positiveSegments: 19, negativeSegments: 6 3712: positiveSegments: 10, negativeSegments: 6 3713: positiveSegments: 31, negativeSegments: 0 3719: positiveSegments: 18, negativeSegments: 3 3722: positiveSegments: 12, negativeSegments: 6 3723: positiveSegments: 1, negativeSegments: 3 3724: positiveSegments: 5, negativeSegments: 0 3725: positiveSegments: 14, negativeSegments: 3 3727: positiveSegments: 2, negativeSegments: 3 3728: positiveSegments: 11, negativeSegments: 3 3729: positiveSegments: 0, negativeSegments: 9 3730: positiveSegments: 8, negativeSegments: 3 3732: positiveSegments: 4, negativeSegments: 3 3736: positiveSegments: 10, negativeSegments: 0 3737: positiveSegments: 16, negativeSegments: 6 3739: positiveSegments: 15, negativeSegments: 0 3740: positiveSegments: 4, negativeSegments: 3 3743: positiveSegments: 0, negativeSegments: 9 3744: positiveSegments: 28, negativeSegments: 0 3748: positiveSegments: 8, negativeSegments: 3 3749: positiveSegments: 17, negativeSegments: 12 3750: positiveSegments: 0, negativeSegments: 12 3752: positiveSegments: 4, negativeSegments: 18 3753: positiveSegments: 0, negativeSegments: 27 3757: positiveSegments: 0, negativeSegments: 3 3758: positiveSegments: 6, negativeSegments: 3 3761: positiveSegments: 12, negativeSegments: 3 3763: positiveSegments: 2, negativeSegments: 6 3764: positiveSegments: 0, negativeSegments: 9 3768: positiveSegments: 7, negativeSegments: 9 3774: positiveSegments: 10, negativeSegments: 3 3775: positiveSegments: 19, negativeSegments: 6 3776: positiveSegments: 1, negativeSegments: 3 3777: positiveSegments: 8, negativeSegments: 3 3782: positiveSegments: 7, negativeSegments: 3 3783: positiveSegments: 10, negativeSegments: 0 3784: positiveSegments: 12, negativeSegments: 0 3785: positiveSegments: 31, negativeSegments: 9 3789: positiveSegments: 0, negativeSegments: 3 3791: positiveSegments: 7, negativeSegments: 9 3793: positiveSegments: 0, negativeSegments: 3 3798: positiveSegments: 6, negativeSegments: 15 3799: positiveSegments: 4, negativeSegments: 0 3800: positiveSegments: 36, negativeSegments: 3 3802: positiveSegments: 8, negativeSegments: 0 3803: positiveSegments: 12, negativeSegments: 0 3805: positiveSegments: 17, negativeSegments: 6 3810: positiveSegments: 4, negativeSegments: 6 3812: positiveSegments: 14, negativeSegments: 0 3813: positiveSegments: 8, negativeSegments: 6 3814: positiveSegments: 0, negativeSegments: 6 3816: positiveSegments: 15, negativeSegments: 3 3817: positiveSegments: 10, negativeSegments: 6 3818: positiveSegments: 0, negativeSegments: 15 3819: positiveSegments: 13, negativeSegments: 3 3821: exit early, no segments to save 3821: positiveSegments: 0, negativeSegments: 0 3822: positiveSegments: 17, negativeSegments: 0 3823: positiveSegments: 4, negativeSegments: 9 3824: positiveSegments: 37, negativeSegments: 6 3825: positiveSegments: 10, negativeSegments: 0 3828: positiveSegments: 8, negativeSegments: 3 3831: positiveSegments: 7, negativeSegments: 9 3832: positiveSegments: 3, negativeSegments: 18 3835: positiveSegments: 3, negativeSegments: 12 3836: positiveSegments: 17, negativeSegments: 6 3837: positiveSegments: 4, negativeSegments: 9 3839: positiveSegments: 6, negativeSegments: 0 3840: positiveSegments: 8, negativeSegments: 3 3842: positiveSegments: 16, negativeSegments: 18 3843: positiveSegments: 26, negativeSegments: 6 3844: positiveSegments: 10, negativeSegments: 18 3845: positiveSegments: 6, negativeSegments: 6 3846: positiveSegments: 12, negativeSegments: 9 3848: positiveSegments: 6, negativeSegments: 0 3849: positiveSegments: 8, negativeSegments: 3 3850: positiveSegments: 4, negativeSegments: 12 3854: positiveSegments: 51, negativeSegments: 6 3855: positiveSegments: 3, negativeSegments: 6 3857: positiveSegments: 14, negativeSegments: 3 3858: exit early, no segments to save 3858: positiveSegments: 0, negativeSegments: 0 3859: positiveSegments: 2, negativeSegments: 0 3863: positiveSegments: 13, negativeSegments: 15 3864: positiveSegments: 9, negativeSegments: 3 3868: positiveSegments: 5, negativeSegments: 3 3870: positiveSegments: 28, negativeSegments: 12 3877: positiveSegments: 11, negativeSegments: 9 3878: positiveSegments: 22, negativeSegments: 6 3879: positiveSegments: 25, negativeSegments: 0 3881: positiveSegments: 4, negativeSegments: 0 3886: positiveSegments: 12, negativeSegments: 3 count processed: 1900, current case index: 3887 3887: positiveSegments: 5, negativeSegments: 6 3888: positiveSegments: 8, negativeSegments: 6 3889: positiveSegments: 0, negativeSegments: 12 3890: positiveSegments: 10, negativeSegments: 0 3891: positiveSegments: 0, negativeSegments: 6 3893: positiveSegments: 23, negativeSegments: 3 3894: positiveSegments: 2, negativeSegments: 3 3895: positiveSegments: 14, negativeSegments: 9 3897: positiveSegments: 0, negativeSegments: 6 3898: positiveSegments: 8, negativeSegments: 0 3902: positiveSegments: 8, negativeSegments: 6 3904: positiveSegments: 19, negativeSegments: 3 3906: positiveSegments: 8, negativeSegments: 3 3910: positiveSegments: 0, negativeSegments: 3 3912: positiveSegments: 0, negativeSegments: 3 3913: positiveSegments: 21, negativeSegments: 6 3919: positiveSegments: 13, negativeSegments: 3 3922: positiveSegments: 4, negativeSegments: 0 3925: positiveSegments: 0, negativeSegments: 9 3928: positiveSegments: 6, negativeSegments: 0 3929: positiveSegments: 11, negativeSegments: 3 3930: positiveSegments: 25, negativeSegments: 3 3931: positiveSegments: 12, negativeSegments: 3 3934: positiveSegments: 3, negativeSegments: 3 3935: positiveSegments: 9, negativeSegments: 0 3936: positiveSegments: 13, negativeSegments: 0 3937: positiveSegments: 8, negativeSegments: 0 3938: positiveSegments: 9, negativeSegments: 3 3944: positiveSegments: 10, negativeSegments: 15 3949: positiveSegments: 8, negativeSegments: 6 3950: positiveSegments: 11, negativeSegments: 15 3955: positiveSegments: 30, negativeSegments: 0 3958: positiveSegments: 4, negativeSegments: 6 3962: positiveSegments: 15, negativeSegments: 0 3963: positiveSegments: 5, negativeSegments: 9 3967: positiveSegments: 4, negativeSegments: 6 3968: positiveSegments: 10, negativeSegments: 3 3971: nothing saved, all segments filtered 3971: positiveSegments: 1, negativeSegments: 0 3972: positiveSegments: 11, negativeSegments: 0 3973: positiveSegments: 4, negativeSegments: 12 3974: positiveSegments: 0, negativeSegments: 15 3975: positiveSegments: 4, negativeSegments: 9 3976: positiveSegments: 8, negativeSegments: 12 3978: positiveSegments: 4, negativeSegments: 0 3980: exit early, no segments to save 3980: positiveSegments: 0, negativeSegments: 0 3981: positiveSegments: 6, negativeSegments: 0 3986: positiveSegments: 4, negativeSegments: 3 3987: positiveSegments: 3, negativeSegments: 6 3988: positiveSegments: 14, negativeSegments: 3 3990: positiveSegments: 2, negativeSegments: 0 3991: positiveSegments: 0, negativeSegments: 12 3992: exit early, no segments to save 3992: positiveSegments: 0, negativeSegments: 0 3993: positiveSegments: 14, negativeSegments: 9 3994: positiveSegments: 0, negativeSegments: 24 3998: positiveSegments: 0, negativeSegments: 21 3999: positiveSegments: 0, negativeSegments: 3 4000: positiveSegments: 6, negativeSegments: 0 4004: positiveSegments: 0, negativeSegments: 6 4005: positiveSegments: 15, negativeSegments: 9 4007: positiveSegments: 8, negativeSegments: 3 4009: positiveSegments: 1, negativeSegments: 0 4010: positiveSegments: 7, negativeSegments: 3 4011: positiveSegments: 0, negativeSegments: 3 4012: positiveSegments: 4, negativeSegments: 3 4013: positiveSegments: 18, negativeSegments: 9 4016: positiveSegments: 4, negativeSegments: 0 4017: positiveSegments: 5, negativeSegments: 3 4020: positiveSegments: 0, negativeSegments: 6 4022: positiveSegments: 5, negativeSegments: 0 4024: positiveSegments: 6, negativeSegments: 0 4026: positiveSegments: 4, negativeSegments: 21 4027: positiveSegments: 0, negativeSegments: 12 4028: positiveSegments: 0, negativeSegments: 6 4030: positiveSegments: 0, negativeSegments: 9 4032: positiveSegments: 8, negativeSegments: 9 4033: positiveSegments: 16, negativeSegments: 0 4034: positiveSegments: 9, negativeSegments: 0 4035: positiveSegments: 7, negativeSegments: 3 4036: positiveSegments: 9, negativeSegments: 9 4037: exit early, no segments to save 4037: positiveSegments: 0, negativeSegments: 0 4040: positiveSegments: 14, negativeSegments: 3 4042: positiveSegments: 0, negativeSegments: 9 4043: positiveSegments: 27, negativeSegments: 9 4045: positiveSegments: 13, negativeSegments: 0 4046: positiveSegments: 5, negativeSegments: 3 4047: exit early, no segments to save 4047: positiveSegments: 0, negativeSegments: 0 4048: positiveSegments: 7, negativeSegments: 0 4050: positiveSegments: 2, negativeSegments: 12 4054: positiveSegments: 26, negativeSegments: 6 4060: positiveSegments: 8, negativeSegments: 15 4062: positiveSegments: 4, negativeSegments: 0 4066: positiveSegments: 22, negativeSegments: 3 4067: positiveSegments: 3, negativeSegments: 3 4069: exit early, no segments to save 4069: positiveSegments: 0, negativeSegments: 0 4070: positiveSegments: 20, negativeSegments: 0 4072: positiveSegments: 21, negativeSegments: 0 4073: positiveSegments: 0, negativeSegments: 6 4074: positiveSegments: 0, negativeSegments: 15 4077: positiveSegments: 4, negativeSegments: 3 4083: positiveSegments: 0, negativeSegments: 6 count processed: 2000, current case index: 4091 4091: positiveSegments: 9, negativeSegments: 12 4093: positiveSegments: 27, negativeSegments: 6 4098: positiveSegments: 4, negativeSegments: 15 4100: positiveSegments: 1, negativeSegments: 0 4101: positiveSegments: 4, negativeSegments: 3 4106: positiveSegments: 4, negativeSegments: 3 4107: positiveSegments: 4, negativeSegments: 0 4109: positiveSegments: 8, negativeSegments: 0 4111: positiveSegments: 10, negativeSegments: 0 4112: positiveSegments: 8, negativeSegments: 0 4114: positiveSegments: 27, negativeSegments: 6 4115: positiveSegments: 4, negativeSegments: 9 4116: positiveSegments: 8, negativeSegments: 3 4120: positiveSegments: 0, negativeSegments: 3 4122: exit early, no segments to save 4122: positiveSegments: 0, negativeSegments: 0 4127: positiveSegments: 8, negativeSegments: 3 4133: positiveSegments: 0, negativeSegments: 12 4137: positiveSegments: 22, negativeSegments: 3 4140: positiveSegments: 41, negativeSegments: 9 4142: exit early, no segments to save 4142: positiveSegments: 0, negativeSegments: 0 4143: positiveSegments: 29, negativeSegments: 12 4144: positiveSegments: 22, negativeSegments: 3 4146: positiveSegments: 4, negativeSegments: 18 4148: positiveSegments: 8, negativeSegments: 9 4149: positiveSegments: 17, negativeSegments: 0 4150: positiveSegments: 17, negativeSegments: 0 4152: positiveSegments: 6, negativeSegments: 0 4155: positiveSegments: 6, negativeSegments: 3 4162: positiveSegments: 11, negativeSegments: 9 4165: exit early, no segments to save 4165: positiveSegments: 0, negativeSegments: 0 4166: positiveSegments: 30, negativeSegments: 21 4167: positiveSegments: 21, negativeSegments: 0 4168: positiveSegments: 7, negativeSegments: 0 4172: positiveSegments: 9, negativeSegments: 6 4173: positiveSegments: 4, negativeSegments: 3 4177: positiveSegments: 6, negativeSegments: 0 4179: positiveSegments: 26, negativeSegments: 0 4181: positiveSegments: 4, negativeSegments: 3 4183: positiveSegments: 4, negativeSegments: 0 4186: positiveSegments: 7, negativeSegments: 3 4187: exit early, no segments to save 4187: positiveSegments: 0, negativeSegments: 0 4189: positiveSegments: 4, negativeSegments: 9 4191: positiveSegments: 12, negativeSegments: 15 4195: positiveSegments: 0, negativeSegments: 3 4198: positiveSegments: 0, negativeSegments: 3 4201: positiveSegments: 8, negativeSegments: 0 4202: positiveSegments: 8, negativeSegments: 9 4203: positiveSegments: 4, negativeSegments: 0 4206: positiveSegments: 16, negativeSegments: 0 4207: positiveSegments: 8, negativeSegments: 18 4208: positiveSegments: 8, negativeSegments: 0 4210: positiveSegments: 13, negativeSegments: 3 4211: positiveSegments: 18, negativeSegments: 3 4212: positiveSegments: 1, negativeSegments: 9 4213: positiveSegments: 0, negativeSegments: 3 4214: positiveSegments: 1, negativeSegments: 15 4216: positiveSegments: 0, negativeSegments: 18 4219: exit early, no segments to save 4219: positiveSegments: 0, negativeSegments: 0 4222: positiveSegments: 4, negativeSegments: 12 4223: positiveSegments: 2, negativeSegments: 3 4225: positiveSegments: 22, negativeSegments: 12 4227: positiveSegments: 4, negativeSegments: 3 4233: positiveSegments: 0, negativeSegments: 9 4236: positiveSegments: 12, negativeSegments: 3 4238: positiveSegments: 10, negativeSegments: 9 4240: positiveSegments: 32, negativeSegments: 0 4241: positiveSegments: 0, negativeSegments: 3 4242: positiveSegments: 8, negativeSegments: 3 4245: positiveSegments: 26, negativeSegments: 0 4247: positiveSegments: 0, negativeSegments: 3 4249: positiveSegments: 18, negativeSegments: 3 4251: positiveSegments: 17, negativeSegments: 0 4252: positiveSegments: 0, negativeSegments: 12 4253: positiveSegments: 4, negativeSegments: 0 4254: positiveSegments: 4, negativeSegments: 3 4255: positiveSegments: 6, negativeSegments: 3 4256: positiveSegments: 46, negativeSegments: 6 4258: positiveSegments: 17, negativeSegments: 3 4259: positiveSegments: 5, negativeSegments: 6 4262: positiveSegments: 7, negativeSegments: 0 4264: positiveSegments: 18, negativeSegments: 6 4265: positiveSegments: 13, negativeSegments: 0 4268: exit early, no segments to save 4268: positiveSegments: 0, negativeSegments: 0 4269: positiveSegments: 17, negativeSegments: 6 4272: positiveSegments: 23, negativeSegments: 3 4277: exit early, no segments to save 4277: positiveSegments: 0, negativeSegments: 0 4278: positiveSegments: 18, negativeSegments: 3 4279: positiveSegments: 5, negativeSegments: 3 4280: positiveSegments: 11, negativeSegments: 3 4281: positiveSegments: 2, negativeSegments: 3 4282: positiveSegments: 14, negativeSegments: 3 4283: positiveSegments: 28, negativeSegments: 0 4284: positiveSegments: 15, negativeSegments: 9 4286: positiveSegments: 4, negativeSegments: 3 4287: positiveSegments: 0, negativeSegments: 9 4289: positiveSegments: 36, negativeSegments: 6 4290: positiveSegments: 4, negativeSegments: 0 4292: positiveSegments: 8, negativeSegments: 15 4293: positiveSegments: 8, negativeSegments: 0 4294: positiveSegments: 0, negativeSegments: 9 count processed: 2100, current case index: 4296 4296: positiveSegments: 4, negativeSegments: 6 4302: positiveSegments: 25, negativeSegments: 6 4304: positiveSegments: 36, negativeSegments: 0 4305: positiveSegments: 13, negativeSegments: 0 4307: positiveSegments: 6, negativeSegments: 3 4308: positiveSegments: 4, negativeSegments: 3 4309: positiveSegments: 11, negativeSegments: 3 4310: positiveSegments: 4, negativeSegments: 12 4314: positiveSegments: 3, negativeSegments: 9 4316: positiveSegments: 20, negativeSegments: 3 4317: positiveSegments: 13, negativeSegments: 0 4320: positiveSegments: 12, negativeSegments: 6 4322: positiveSegments: 4, negativeSegments: 9 4325: positiveSegments: 3, negativeSegments: 3 4326: positiveSegments: 8, negativeSegments: 3 4327: positiveSegments: 13, negativeSegments: 0 4328: exit early, no segments to save 4328: positiveSegments: 0, negativeSegments: 0 4332: positiveSegments: 13, negativeSegments: 0 4333: positiveSegments: 14, negativeSegments: 3 4335: positiveSegments: 3, negativeSegments: 12 4339: positiveSegments: 7, negativeSegments: 3 4341: positiveSegments: 17, negativeSegments: 12 4345: positiveSegments: 10, negativeSegments: 0 4347: positiveSegments: 26, negativeSegments: 3 4350: positiveSegments: 0, negativeSegments: 3 4352: positiveSegments: 2, negativeSegments: 0 4354: positiveSegments: 12, negativeSegments: 3 4356: positiveSegments: 6, negativeSegments: 9 4362: positiveSegments: 4, negativeSegments: 0 4364: positiveSegments: 9, negativeSegments: 3 4367: positiveSegments: 5, negativeSegments: 12 4368: positiveSegments: 16, negativeSegments: 6 4371: exit early, no segments to save 4371: positiveSegments: 0, negativeSegments: 0 4375: positiveSegments: 0, negativeSegments: 9 4377: positiveSegments: 8, negativeSegments: 15 4380: positiveSegments: 4, negativeSegments: 0 4382: positiveSegments: 14, negativeSegments: 3 4383: positiveSegments: 19, negativeSegments: 6 4385: positiveSegments: 7, negativeSegments: 6 4387: positiveSegments: 0, negativeSegments: 3 4388: positiveSegments: 12, negativeSegments: 0 4389: positiveSegments: 0, negativeSegments: 6 4390: positiveSegments: 12, negativeSegments: 6 4392: positiveSegments: 15, negativeSegments: 3 4396: positiveSegments: 0, negativeSegments: 3 4398: positiveSegments: 26, negativeSegments: 0 4400: positiveSegments: 4, negativeSegments: 6 4401: positiveSegments: 10, negativeSegments: 6 4402: positiveSegments: 7, negativeSegments: 9 4405: positiveSegments: 45, negativeSegments: 0 4406: positiveSegments: 11, negativeSegments: 0 4408: positiveSegments: 5, negativeSegments: 0 4409: positiveSegments: 8, negativeSegments: 9 4411: positiveSegments: 14, negativeSegments: 0 4414: exit early, no segments to save 4414: positiveSegments: 0, negativeSegments: 0 4417: positiveSegments: 5, negativeSegments: 3 4424: positiveSegments: 0, negativeSegments: 9 4425: exit early, no segments to save 4425: positiveSegments: 0, negativeSegments: 0 4426: positiveSegments: 0, negativeSegments: 6 4428: positiveSegments: 4, negativeSegments: 0 4429: positiveSegments: 18, negativeSegments: 6 4430: positiveSegments: 2, negativeSegments: 6 4432: positiveSegments: 8, negativeSegments: 6 4433: exit early, no segments to save 4433: positiveSegments: 0, negativeSegments: 0 4435: positiveSegments: 4, negativeSegments: 0 4437: positiveSegments: 7, negativeSegments: 3 4439: positiveSegments: 13, negativeSegments: 0 4443: positiveSegments: 0, negativeSegments: 6 4449: positiveSegments: 0, negativeSegments: 6 4451: positiveSegments: 12, negativeSegments: 6 4453: positiveSegments: 14, negativeSegments: 0 4456: positiveSegments: 7, negativeSegments: 12 4457: positiveSegments: 8, negativeSegments: 9 4458: positiveSegments: 7, negativeSegments: 0 4459: exit early, no segments to save 4459: positiveSegments: 0, negativeSegments: 0 4461: positiveSegments: 9, negativeSegments: 15 4462: positiveSegments: 11, negativeSegments: 0 4463: positiveSegments: 5, negativeSegments: 12 4464: exit early, no segments to save 4464: positiveSegments: 0, negativeSegments: 0 4466: exit early, no segments to save 4466: positiveSegments: 0, negativeSegments: 0 4470: positiveSegments: 4, negativeSegments: 6 4472: positiveSegments: 13, negativeSegments: 0 4474: positiveSegments: 0, negativeSegments: 9 4475: positiveSegments: 8, negativeSegments: 9 4476: positiveSegments: 19, negativeSegments: 3 4477: positiveSegments: 5, negativeSegments: 0 4478: positiveSegments: 4, negativeSegments: 12 4480: positiveSegments: 4, negativeSegments: 3 4481: positiveSegments: 26, negativeSegments: 6 4483: positiveSegments: 0, negativeSegments: 6 4485: positiveSegments: 1, negativeSegments: 0 4489: positiveSegments: 8, negativeSegments: 9 4490: positiveSegments: 9, negativeSegments: 0 4496: positiveSegments: 17, negativeSegments: 9 4497: positiveSegments: 5, negativeSegments: 0 4498: positiveSegments: 2, negativeSegments: 9 4501: exit early, no segments to save 4501: positiveSegments: 0, negativeSegments: 0 4502: positiveSegments: 8, negativeSegments: 3 4503: positiveSegments: 0, negativeSegments: 3 4504: positiveSegments: 54, negativeSegments: 0 count processed: 2200, current case index: 4509 4509: positiveSegments: 4, negativeSegments: 9 4510: positiveSegments: 21, negativeSegments: 0 4515: positiveSegments: 7, negativeSegments: 0 4519: exit early, no segments to save 4519: positiveSegments: 0, negativeSegments: 0 4520: positiveSegments: 4, negativeSegments: 3 4522: positiveSegments: 0, negativeSegments: 3 4525: positiveSegments: 0, negativeSegments: 9 4530: positiveSegments: 8, negativeSegments: 6 4536: positiveSegments: 2, negativeSegments: 3 4538: positiveSegments: 6, negativeSegments: 3 4540: positiveSegments: 0, negativeSegments: 12 4541: positiveSegments: 4, negativeSegments: 0 4546: positiveSegments: 7, negativeSegments: 3 4547: positiveSegments: 0, negativeSegments: 6 4549: positiveSegments: 10, negativeSegments: 3 4550: positiveSegments: 2, negativeSegments: 9 4557: positiveSegments: 4, negativeSegments: 0 4559: positiveSegments: 12, negativeSegments: 0 4561: positiveSegments: 8, negativeSegments: 0 4564: positiveSegments: 4, negativeSegments: 3 4568: positiveSegments: 15, negativeSegments: 6 4569: positiveSegments: 1, negativeSegments: 3 4572: positiveSegments: 3, negativeSegments: 0 4573: positiveSegments: 40, negativeSegments: 3 4574: positiveSegments: 28, negativeSegments: 12 4576: positiveSegments: 4, negativeSegments: 6 4577: positiveSegments: 0, negativeSegments: 3 4578: positiveSegments: 5, negativeSegments: 0 4579: positiveSegments: 5, negativeSegments: 0 4581: positiveSegments: 4, negativeSegments: 3 4584: positiveSegments: 0, negativeSegments: 9 4589: positiveSegments: 8, negativeSegments: 9 4591: positiveSegments: 19, negativeSegments: 0 4596: positiveSegments: 0, negativeSegments: 6 4597: positiveSegments: 16, negativeSegments: 0 4599: positiveSegments: 10, negativeSegments: 0 4600: positiveSegments: 11, negativeSegments: 6 4602: positiveSegments: 15, negativeSegments: 0 4603: positiveSegments: 4, negativeSegments: 9 4604: positiveSegments: 18, negativeSegments: 12 4605: exit early, no segments to save 4605: positiveSegments: 0, negativeSegments: 0 4607: positiveSegments: 16, negativeSegments: 0 4609: positiveSegments: 0, negativeSegments: 24 4612: positiveSegments: 11, negativeSegments: 6 4613: exit early, no segments to save 4613: positiveSegments: 0, negativeSegments: 0 4616: positiveSegments: 4, negativeSegments: 0 4617: positiveSegments: 9, negativeSegments: 9 4618: positiveSegments: 10, negativeSegments: 6 4619: positiveSegments: 8, negativeSegments: 6 4620: positiveSegments: 6, negativeSegments: 0 4621: positiveSegments: 14, negativeSegments: 6 4622: positiveSegments: 4, negativeSegments: 0 4626: positiveSegments: 8, negativeSegments: 3 4627: positiveSegments: 13, negativeSegments: 0 4631: positiveSegments: 3, negativeSegments: 3 4632: positiveSegments: 18, negativeSegments: 6 4635: positiveSegments: 8, negativeSegments: 0 4639: positiveSegments: 6, negativeSegments: 6 4640: positiveSegments: 4, negativeSegments: 0 4644: positiveSegments: 18, negativeSegments: 15 4646: positiveSegments: 9, negativeSegments: 3 4648: exit early, no segments to save 4648: positiveSegments: 0, negativeSegments: 0 4650: positiveSegments: 3, negativeSegments: 0 4652: positiveSegments: 31, negativeSegments: 6 4653: positiveSegments: 11, negativeSegments: 3 4654: positiveSegments: 9, negativeSegments: 9 4655: positiveSegments: 4, negativeSegments: 0 4656: positiveSegments: 9, negativeSegments: 0 4657: positiveSegments: 26, negativeSegments: 0 4658: positiveSegments: 8, negativeSegments: 6 4660: positiveSegments: 4, negativeSegments: 6 4662: positiveSegments: 16, negativeSegments: 0 4665: positiveSegments: 19, negativeSegments: 3 4666: positiveSegments: 5, negativeSegments: 15 4670: positiveSegments: 48, negativeSegments: 0 4673: exit early, no segments to save 4673: positiveSegments: 0, negativeSegments: 0 4678: positiveSegments: 0, negativeSegments: 6 4683: positiveSegments: 9, negativeSegments: 0 4684: positiveSegments: 25, negativeSegments: 9 4686: positiveSegments: 11, negativeSegments: 3 4690: exit early, no segments to save 4690: positiveSegments: 0, negativeSegments: 0 4695: positiveSegments: 6, negativeSegments: 0 4700: positiveSegments: 10, negativeSegments: 3 4702: exit early, no segments to save 4702: positiveSegments: 0, negativeSegments: 0 4703: exit early, no segments to save 4703: positiveSegments: 0, negativeSegments: 0 4705: positiveSegments: 4, negativeSegments: 0 4706: positiveSegments: 0, negativeSegments: 9 4711: positiveSegments: 8, negativeSegments: 3 4713: positiveSegments: 0, negativeSegments: 12 4714: positiveSegments: 6, negativeSegments: 3 4715: positiveSegments: 20, negativeSegments: 9 4716: positiveSegments: 4, negativeSegments: 3 4717: positiveSegments: 0, negativeSegments: 6 4718: positiveSegments: 15, negativeSegments: 3 4721: positiveSegments: 28, negativeSegments: 0 4724: positiveSegments: 0, negativeSegments: 3 4726: positiveSegments: 5, negativeSegments: 3 4727: nothing saved, all segments filtered 4727: positiveSegments: 1, negativeSegments: 0 4729: positiveSegments: 4, negativeSegments: 6 4731: positiveSegments: 8, negativeSegments: 12 count processed: 2300, current case index: 4732 4732: positiveSegments: 1, negativeSegments: 15 4733: exit early, no segments to save 4733: positiveSegments: 0, negativeSegments: 0 4739: positiveSegments: 13, negativeSegments: 0 4741: exit early, no segments to save 4741: positiveSegments: 0, negativeSegments: 0 4743: positiveSegments: 31, negativeSegments: 3 4744: positiveSegments: 6, negativeSegments: 0 4745: positiveSegments: 4, negativeSegments: 9 4746: positiveSegments: 12, negativeSegments: 6 4749: positiveSegments: 22, negativeSegments: 3 4750: positiveSegments: 0, negativeSegments: 12 4752: positiveSegments: 7, negativeSegments: 9 4755: positiveSegments: 4, negativeSegments: 0 4757: positiveSegments: 18, negativeSegments: 15 4759: positiveSegments: 28, negativeSegments: 9 4761: positiveSegments: 6, negativeSegments: 0 4763: positiveSegments: 6, negativeSegments: 0 4764: positiveSegments: 12, negativeSegments: 12 4767: positiveSegments: 4, negativeSegments: 3 4768: positiveSegments: 7, negativeSegments: 3 4769: exit early, no segments to save 4769: positiveSegments: 0, negativeSegments: 0 4771: positiveSegments: 5, negativeSegments: 0 4773: positiveSegments: 13, negativeSegments: 6 4775: positiveSegments: 19, negativeSegments: 3 4777: positiveSegments: 10, negativeSegments: 0 4778: positiveSegments: 17, negativeSegments: 0 4779: positiveSegments: 27, negativeSegments: 0 4780: positiveSegments: 8, negativeSegments: 3 4781: positiveSegments: 4, negativeSegments: 21 4783: positiveSegments: 4, negativeSegments: 0 4784: exit early, no segments to save 4784: positiveSegments: 0, negativeSegments: 0 4785: positiveSegments: 0, negativeSegments: 6 4786: positiveSegments: 7, negativeSegments: 0 4788: positiveSegments: 13, negativeSegments: 3 4789: positiveSegments: 8, negativeSegments: 6 4790: positiveSegments: 2, negativeSegments: 0 4792: positiveSegments: 4, negativeSegments: 9 4794: positiveSegments: 0, negativeSegments: 3 4798: positiveSegments: 10, negativeSegments: 9 4800: positiveSegments: 26, negativeSegments: 3 4801: positiveSegments: 29, negativeSegments: 9 4802: positiveSegments: 9, negativeSegments: 0 4803: positiveSegments: 4, negativeSegments: 3 4805: positiveSegments: 4, negativeSegments: 0 4806: positiveSegments: 0, negativeSegments: 9 4808: positiveSegments: 18, negativeSegments: 3 4809: positiveSegments: 16, negativeSegments: 6 4813: positiveSegments: 4, negativeSegments: 3 4816: positiveSegments: 17, negativeSegments: 0 4817: positiveSegments: 45, negativeSegments: 3 4818: positiveSegments: 0, negativeSegments: 3 4820: positiveSegments: 16, negativeSegments: 0 4823: positiveSegments: 5, negativeSegments: 6 4825: positiveSegments: 10, negativeSegments: 0 4826: positiveSegments: 8, negativeSegments: 3 4828: positiveSegments: 22, negativeSegments: 0 4830: positiveSegments: 4, negativeSegments: 0 4831: positiveSegments: 12, negativeSegments: 3 4834: nothing saved, all segments filtered 4834: positiveSegments: 3, negativeSegments: 0 4835: positiveSegments: 11, negativeSegments: 6 4836: nothing saved, all segments filtered 4836: positiveSegments: 11, negativeSegments: 0 4837: positiveSegments: 3, negativeSegments: 3 4839: positiveSegments: 16, negativeSegments: 6 4841: positiveSegments: 7, negativeSegments: 0 4843: positiveSegments: 3, negativeSegments: 6 4844: positiveSegments: 0, negativeSegments: 3 4847: positiveSegments: 0, negativeSegments: 3 4851: positiveSegments: 23, negativeSegments: 0 4853: positiveSegments: 11, negativeSegments: 9 4857: positiveSegments: 0, negativeSegments: 3 4858: positiveSegments: 22, negativeSegments: 3 4859: positiveSegments: 0, negativeSegments: 6 4860: positiveSegments: 3, negativeSegments: 0 4861: positiveSegments: 20, negativeSegments: 0 4865: positiveSegments: 3, negativeSegments: 9 4869: positiveSegments: 16, negativeSegments: 3 4871: positiveSegments: 34, negativeSegments: 3 4872: positiveSegments: 4, negativeSegments: 6 4874: positiveSegments: 0, negativeSegments: 6 4875: positiveSegments: 5, negativeSegments: 3 4879: positiveSegments: 8, negativeSegments: 3 4880: positiveSegments: 8, negativeSegments: 3 4882: positiveSegments: 4, negativeSegments: 3 4883: positiveSegments: 4, negativeSegments: 0 4886: positiveSegments: 4, negativeSegments: 0 4887: positiveSegments: 21, negativeSegments: 15 4893: positiveSegments: 27, negativeSegments: 0 4894: positiveSegments: 13, negativeSegments: 6 4897: positiveSegments: 16, negativeSegments: 3 4899: positiveSegments: 0, negativeSegments: 12 4901: positiveSegments: 3, negativeSegments: 0 4902: positiveSegments: 13, negativeSegments: 3 4903: positiveSegments: 0, negativeSegments: 3 4904: positiveSegments: 0, negativeSegments: 3 4907: positiveSegments: 6, negativeSegments: 6 4911: positiveSegments: 14, negativeSegments: 6 4912: positiveSegments: 12, negativeSegments: 15 4913: positiveSegments: 0, negativeSegments: 3 4914: positiveSegments: 7, negativeSegments: 0 4916: positiveSegments: 18, negativeSegments: 0 4925: positiveSegments: 0, negativeSegments: 6 count processed: 2400, current case index: 4929 4929: positiveSegments: 10, negativeSegments: 3 4932: positiveSegments: 0, negativeSegments: 6 4933: positiveSegments: 21, negativeSegments: 0 4934: positiveSegments: 28, negativeSegments: 12 4935: positiveSegments: 2, negativeSegments: 6 4936: exit early, no segments to save 4936: positiveSegments: 0, negativeSegments: 0 4938: exit early, no segments to save 4938: positiveSegments: 0, negativeSegments: 0 4939: positiveSegments: 4, negativeSegments: 9 4941: nothing saved, all segments filtered 4941: positiveSegments: 1, negativeSegments: 0 4942: positiveSegments: 18, negativeSegments: 3 4943: positiveSegments: 0, negativeSegments: 9 4946: positiveSegments: 4, negativeSegments: 0 4947: positiveSegments: 4, negativeSegments: 18 4949: positiveSegments: 0, negativeSegments: 6 4951: positiveSegments: 8, negativeSegments: 6 4953: positiveSegments: 21, negativeSegments: 9 4954: positiveSegments: 11, negativeSegments: 6 4957: positiveSegments: 15, negativeSegments: 9 4958: positiveSegments: 18, negativeSegments: 6 4959: positiveSegments: 35, negativeSegments: 3 4964: positiveSegments: 4, negativeSegments: 0 4965: positiveSegments: 0, negativeSegments: 3 4966: positiveSegments: 24, negativeSegments: 3 4971: positiveSegments: 13, negativeSegments: 0 4973: positiveSegments: 4, negativeSegments: 6 4974: positiveSegments: 10, negativeSegments: 6 4976: positiveSegments: 9, negativeSegments: 9 4977: positiveSegments: 8, negativeSegments: 0 4982: positiveSegments: 7, negativeSegments: 3 4985: nothing saved, all segments filtered 4985: positiveSegments: 1, negativeSegments: 0 4987: positiveSegments: 9, negativeSegments: 12 4989: positiveSegments: 6, negativeSegments: 3 4991: positiveSegments: 0, negativeSegments: 3 4992: positiveSegments: 12, negativeSegments: 0 4995: positiveSegments: 7, negativeSegments: 15 4997: positiveSegments: 34, negativeSegments: 3 4998: positiveSegments: 7, negativeSegments: 0 4999: positiveSegments: 11, negativeSegments: 12 5001: positiveSegments: 3, negativeSegments: 0 5004: positiveSegments: 26, negativeSegments: 0 5005: positiveSegments: 16, negativeSegments: 0 5006: positiveSegments: 24, negativeSegments: 3 5008: positiveSegments: 0, negativeSegments: 6 5010: positiveSegments: 12, negativeSegments: 15 5014: positiveSegments: 0, negativeSegments: 15 5015: positiveSegments: 2, negativeSegments: 9 5018: positiveSegments: 13, negativeSegments: 0 5019: positiveSegments: 19, negativeSegments: 15 5021: positiveSegments: 6, negativeSegments: 6 5024: positiveSegments: 7, negativeSegments: 3 5027: positiveSegments: 8, negativeSegments: 0 5029: positiveSegments: 0, negativeSegments: 3 5031: positiveSegments: 6, negativeSegments: 0 5033: positiveSegments: 4, negativeSegments: 3 5036: positiveSegments: 25, negativeSegments: 0 5040: positiveSegments: 7, negativeSegments: 3 5043: positiveSegments: 14, negativeSegments: 6 5044: positiveSegments: 20, negativeSegments: 6 5045: positiveSegments: 12, negativeSegments: 3 5046: positiveSegments: 8, negativeSegments: 0 5052: positiveSegments: 15, negativeSegments: 0 5059: positiveSegments: 24, negativeSegments: 0 5060: positiveSegments: 0, negativeSegments: 18 5061: positiveSegments: 3, negativeSegments: 0 5068: positiveSegments: 8, negativeSegments: 3 5070: positiveSegments: 10, negativeSegments: 0 5072: positiveSegments: 0, negativeSegments: 12 5077: positiveSegments: 4, negativeSegments: 6 5079: positiveSegments: 0, negativeSegments: 6 5080: positiveSegments: 4, negativeSegments: 0 5084: positiveSegments: 13, negativeSegments: 0 5089: positiveSegments: 4, negativeSegments: 9 5090: positiveSegments: 11, negativeSegments: 3 5091: positiveSegments: 0, negativeSegments: 9 5092: positiveSegments: 4, negativeSegments: 3 5093: positiveSegments: 6, negativeSegments: 6 5099: positiveSegments: 28, negativeSegments: 12 5100: positiveSegments: 29, negativeSegments: 0 5104: positiveSegments: 3, negativeSegments: 6 5106: positiveSegments: 4, negativeSegments: 15 5108: positiveSegments: 15, negativeSegments: 3 5109: positiveSegments: 14, negativeSegments: 3 5110: positiveSegments: 13, negativeSegments: 0 5111: positiveSegments: 4, negativeSegments: 9 5112: positiveSegments: 4, negativeSegments: 9 5116: positiveSegments: 4, negativeSegments: 0 5117: positiveSegments: 2, negativeSegments: 0 5118: positiveSegments: 17, negativeSegments: 0 5119: positiveSegments: 50, negativeSegments: 3 5122: positiveSegments: 3, negativeSegments: 6 5124: positiveSegments: 2, negativeSegments: 3 5125: positiveSegments: 4, negativeSegments: 0 5126: positiveSegments: 10, negativeSegments: 3 5130: exit early, no segments to save 5130: positiveSegments: 0, negativeSegments: 0 5131: positiveSegments: 20, negativeSegments: 3 5135: positiveSegments: 13, negativeSegments: 9 5137: positiveSegments: 4, negativeSegments: 6 5139: positiveSegments: 26, negativeSegments: 0 5140: positiveSegments: 0, negativeSegments: 9 5141: positiveSegments: 2, negativeSegments: 0 count processed: 2500, current case index: 5142 5142: positiveSegments: 3, negativeSegments: 0 5143: positiveSegments: 0, negativeSegments: 3 5148: exit early, no segments to save 5148: positiveSegments: 0, negativeSegments: 0 5149: positiveSegments: 9, negativeSegments: 0 5150: positiveSegments: 0, negativeSegments: 6 5151: positiveSegments: 10, negativeSegments: 0 5152: positiveSegments: 27, negativeSegments: 6 5153: positiveSegments: 7, negativeSegments: 6 5155: positiveSegments: 4, negativeSegments: 3 5156: positiveSegments: 5, negativeSegments: 0 5157: positiveSegments: 4, negativeSegments: 0 5162: positiveSegments: 4, negativeSegments: 9 5163: positiveSegments: 23, negativeSegments: 9 5164: exit early, no segments to save 5164: positiveSegments: 0, negativeSegments: 0 5166: positiveSegments: 16, negativeSegments: 9 5173: positiveSegments: 40, negativeSegments: 0 5174: positiveSegments: 22, negativeSegments: 0 5175: nothing saved, all segments filtered 5175: positiveSegments: 2, negativeSegments: 0 5178: positiveSegments: 13, negativeSegments: 6 5179: nothing saved, all segments filtered 5179: positiveSegments: 2, negativeSegments: 0 5180: positiveSegments: 4, negativeSegments: 12 5182: positiveSegments: 0, negativeSegments: 6 5183: positiveSegments: 4, negativeSegments: 0 5184: positiveSegments: 12, negativeSegments: 3 5185: positiveSegments: 11, negativeSegments: 0 5187: positiveSegments: 17, negativeSegments: 0 5192: positiveSegments: 0, negativeSegments: 6 5193: positiveSegments: 0, negativeSegments: 6 5194: positiveSegments: 5, negativeSegments: 0 5195: positiveSegments: 4, negativeSegments: 3 5198: positiveSegments: 12, negativeSegments: 6 5201: positiveSegments: 37, negativeSegments: 0 5202: positiveSegments: 2, negativeSegments: 6 5203: positiveSegments: 16, negativeSegments: 12 5204: positiveSegments: 12, negativeSegments: 0 5209: exit early, no segments to save 5209: positiveSegments: 0, negativeSegments: 0 5213: positiveSegments: 8, negativeSegments: 6 5214: positiveSegments: 0, negativeSegments: 3 5217: positiveSegments: 9, negativeSegments: 3 5218: exit early, no segments to save 5218: positiveSegments: 0, negativeSegments: 0 5221: positiveSegments: 11, negativeSegments: 0 5222: positiveSegments: 12, negativeSegments: 12 5223: exit early, no segments to save 5223: positiveSegments: 0, negativeSegments: 0 5224: positiveSegments: 12, negativeSegments: 15 5225: positiveSegments: 15, negativeSegments: 0 5226: positiveSegments: 7, negativeSegments: 0 5229: positiveSegments: 14, negativeSegments: 6 5230: positiveSegments: 7, negativeSegments: 0 5232: positiveSegments: 1, negativeSegments: 12 5234: positiveSegments: 8, negativeSegments: 3 5235: positiveSegments: 6, negativeSegments: 21 5237: positiveSegments: 8, negativeSegments: 0 5245: positiveSegments: 33, negativeSegments: 6 5246: positiveSegments: 12, negativeSegments: 0 5247: positiveSegments: 14, negativeSegments: 9 5251: positiveSegments: 11, negativeSegments: 0 5253: positiveSegments: 6, negativeSegments: 0 5254: positiveSegments: 4, negativeSegments: 3 5255: positiveSegments: 0, negativeSegments: 9 5259: positiveSegments: 11, negativeSegments: 0 5264: positiveSegments: 0, negativeSegments: 21 5265: exit early, no segments to save 5265: positiveSegments: 0, negativeSegments: 0 5266: positiveSegments: 6, negativeSegments: 0 5267: positiveSegments: 8, negativeSegments: 6 5270: positiveSegments: 8, negativeSegments: 0 5271: positiveSegments: 0, negativeSegments: 15 5274: exit early, no segments to save 5274: positiveSegments: 0, negativeSegments: 0 5277: positiveSegments: 4, negativeSegments: 12 5278: positiveSegments: 6, negativeSegments: 9 5280: positiveSegments: 5, negativeSegments: 0 5283: positiveSegments: 2, negativeSegments: 3 5284: positiveSegments: 24, negativeSegments: 0 5285: positiveSegments: 13, negativeSegments: 0 5288: positiveSegments: 4, negativeSegments: 3 5292: positiveSegments: 7, negativeSegments: 0 5295: positiveSegments: 16, negativeSegments: 9 5296: positiveSegments: 5, negativeSegments: 3 5297: positiveSegments: 6, negativeSegments: 0 5298: positiveSegments: 32, negativeSegments: 6 5299: positiveSegments: 8, negativeSegments: 0 5301: exit early, no segments to save 5301: positiveSegments: 0, negativeSegments: 0 5302: positiveSegments: 12, negativeSegments: 0 5304: positiveSegments: 29, negativeSegments: 0 5305: positiveSegments: 21, negativeSegments: 6 5308: positiveSegments: 0, negativeSegments: 18 5309: positiveSegments: 0, negativeSegments: 6 5310: positiveSegments: 15, negativeSegments: 0 5311: positiveSegments: 31, negativeSegments: 3 5314: positiveSegments: 0, negativeSegments: 18 5317: positiveSegments: 11, negativeSegments: 15 5319: positiveSegments: 24, negativeSegments: 9 5321: positiveSegments: 0, negativeSegments: 12 5322: positiveSegments: 10, negativeSegments: 6 5323: positiveSegments: 16, negativeSegments: 3 5324: positiveSegments: 8, negativeSegments: 0 5327: nothing saved, all segments filtered 5327: positiveSegments: 4, negativeSegments: 0 5332: positiveSegments: 0, negativeSegments: 3 5339: nothing saved, all segments filtered 5339: positiveSegments: 2, negativeSegments: 0 5342: positiveSegments: 0, negativeSegments: 6 5344: positiveSegments: 4, negativeSegments: 9 count processed: 2600, current case index: 5346 5346: positiveSegments: 33, negativeSegments: 6 5347: positiveSegments: 21, negativeSegments: 12 5348: exit early, no segments to save 5348: positiveSegments: 0, negativeSegments: 0 5349: positiveSegments: 15, negativeSegments: 6 5352: positiveSegments: 8, negativeSegments: 9 5353: positiveSegments: 8, negativeSegments: 6 5356: positiveSegments: 3, negativeSegments: 0 5359: positiveSegments: 4, negativeSegments: 3 5360: positiveSegments: 12, negativeSegments: 21 5362: positiveSegments: 24, negativeSegments: 3 5363: positiveSegments: 15, negativeSegments: 12 5364: positiveSegments: 3, negativeSegments: 6 5366: exit early, no segments to save 5366: positiveSegments: 0, negativeSegments: 0 5367: positiveSegments: 12, negativeSegments: 3 5368: positiveSegments: 8, negativeSegments: 0 5371: positiveSegments: 4, negativeSegments: 6 5372: positiveSegments: 14, negativeSegments: 3 5379: positiveSegments: 11, negativeSegments: 3 5383: positiveSegments: 7, negativeSegments: 6 5387: positiveSegments: 12, negativeSegments: 6 5388: positiveSegments: 0, negativeSegments: 6 5390: positiveSegments: 6, negativeSegments: 0 5393: positiveSegments: 0, negativeSegments: 3 5394: positiveSegments: 4, negativeSegments: 0 5395: positiveSegments: 0, negativeSegments: 3 5396: positiveSegments: 30, negativeSegments: 18 5397: positiveSegments: 4, negativeSegments: 0 5399: positiveSegments: 12, negativeSegments: 3 5402: positiveSegments: 4, negativeSegments: 0 5403: positiveSegments: 11, negativeSegments: 9 5406: positiveSegments: 0, negativeSegments: 3 5411: positiveSegments: 7, negativeSegments: 0 5415: positiveSegments: 13, negativeSegments: 0 5416: positiveSegments: 46, negativeSegments: 3 5420: positiveSegments: 12, negativeSegments: 3 5421: positiveSegments: 43, negativeSegments: 6 5422: nothing saved, all segments filtered 5422: positiveSegments: 2, negativeSegments: 0 5423: positiveSegments: 31, negativeSegments: 0 5425: positiveSegments: 6, negativeSegments: 12 5427: positiveSegments: 8, negativeSegments: 0 5431: positiveSegments: 0, negativeSegments: 3 5434: positiveSegments: 4, negativeSegments: 6 5436: positiveSegments: 3, negativeSegments: 0 5442: positiveSegments: 8, negativeSegments: 0 5443: positiveSegments: 13, negativeSegments: 6 5446: positiveSegments: 9, negativeSegments: 0 5449: positiveSegments: 6, negativeSegments: 0 5451: positiveSegments: 20, negativeSegments: 3 5453: positiveSegments: 6, negativeSegments: 0 5454: positiveSegments: 22, negativeSegments: 0 5456: positiveSegments: 0, negativeSegments: 6 5458: positiveSegments: 42, negativeSegments: 9 5460: positiveSegments: 5, negativeSegments: 0 5462: positiveSegments: 0, negativeSegments: 6 5463: positiveSegments: 7, negativeSegments: 6 5467: positiveSegments: 14, negativeSegments: 3 5474: positiveSegments: 4, negativeSegments: 6 5475: positiveSegments: 4, negativeSegments: 12 5476: exit early, no segments to save 5476: positiveSegments: 0, negativeSegments: 0 5478: positiveSegments: 7, negativeSegments: 0 5479: positiveSegments: 3, negativeSegments: 6 5480: positiveSegments: 4, negativeSegments: 9 5484: positiveSegments: 4, negativeSegments: 6 5486: positiveSegments: 15, negativeSegments: 3 5487: positiveSegments: 4, negativeSegments: 6 5490: positiveSegments: 18, negativeSegments: 0 5492: positiveSegments: 7, negativeSegments: 6 5495: positiveSegments: 8, negativeSegments: 9 5497: positiveSegments: 28, negativeSegments: 3 5499: positiveSegments: 9, negativeSegments: 6 5500: positiveSegments: 2, negativeSegments: 0 5501: exit early, no segments to save 5501: positiveSegments: 0, negativeSegments: 0 5502: positiveSegments: 9, negativeSegments: 0 5505: positiveSegments: 16, negativeSegments: 3 5507: positiveSegments: 4, negativeSegments: 12 5508: positiveSegments: 0, negativeSegments: 12 5509: positiveSegments: 0, negativeSegments: 12 5511: positiveSegments: 10, negativeSegments: 3 5513: positiveSegments: 8, negativeSegments: 9 5515: positiveSegments: 4, negativeSegments: 12 5516: positiveSegments: 29, negativeSegments: 6 5517: positiveSegments: 4, negativeSegments: 6 5519: positiveSegments: 7, negativeSegments: 9 5520: exit early, no segments to save 5520: positiveSegments: 0, negativeSegments: 0 5524: positiveSegments: 14, negativeSegments: 3 5526: positiveSegments: 1, negativeSegments: 6 5531: positiveSegments: 8, negativeSegments: 9 5533: positiveSegments: 0, negativeSegments: 3 5534: positiveSegments: 42, negativeSegments: 15 5536: positiveSegments: 7, negativeSegments: 3 5537: positiveSegments: 6, negativeSegments: 6 5538: positiveSegments: 11, negativeSegments: 0 5541: positiveSegments: 0, negativeSegments: 3 5544: positiveSegments: 2, negativeSegments: 0 5546: positiveSegments: 7, negativeSegments: 0 5548: positiveSegments: 5, negativeSegments: 3 5552: positiveSegments: 27, negativeSegments: 0 5556: positiveSegments: 4, negativeSegments: 0 5561: positiveSegments: 0, negativeSegments: 3 5562: exit early, no segments to save 5562: positiveSegments: 0, negativeSegments: 0 count processed: 2700, current case index: 5564 5564: positiveSegments: 23, negativeSegments: 0 5566: positiveSegments: 6, negativeSegments: 6 5568: positiveSegments: 18, negativeSegments: 3 5571: exit early, no segments to save 5571: positiveSegments: 0, negativeSegments: 0 5572: positiveSegments: 6, negativeSegments: 9 5573: positiveSegments: 7, negativeSegments: 0 5574: positiveSegments: 14, negativeSegments: 9 5578: positiveSegments: 2, negativeSegments: 3 5582: positiveSegments: 12, negativeSegments: 15 5583: positiveSegments: 15, negativeSegments: 3 5585: positiveSegments: 17, negativeSegments: 3 5587: nothing saved, all segments filtered 5587: positiveSegments: 2, negativeSegments: 0 5589: positiveSegments: 4, negativeSegments: 6 5593: positiveSegments: 16, negativeSegments: 3 5594: positiveSegments: 4, negativeSegments: 3 5595: positiveSegments: 31, negativeSegments: 3 5597: positiveSegments: 7, negativeSegments: 0 5598: positiveSegments: 6, negativeSegments: 0 5600: positiveSegments: 13, negativeSegments: 3 5601: positiveSegments: 11, negativeSegments: 6 5602: positiveSegments: 4, negativeSegments: 12 5603: positiveSegments: 8, negativeSegments: 3 5607: positiveSegments: 36, negativeSegments: 9 5608: positiveSegments: 8, negativeSegments: 0 5610: positiveSegments: 7, negativeSegments: 0 5612: positiveSegments: 18, negativeSegments: 3 5613: positiveSegments: 8, negativeSegments: 9 5614: positiveSegments: 16, negativeSegments: 0 5616: positiveSegments: 0, negativeSegments: 9 5617: nothing saved, all segments filtered 5617: positiveSegments: 4, negativeSegments: 0 5618: positiveSegments: 0, negativeSegments: 15 5620: positiveSegments: 18, negativeSegments: 3 5621: positiveSegments: 10, negativeSegments: 12 5624: positiveSegments: 23, negativeSegments: 3 5626: positiveSegments: 19, negativeSegments: 3 5627: positiveSegments: 5, negativeSegments: 15 5629: positiveSegments: 5, negativeSegments: 0 5630: positiveSegments: 14, negativeSegments: 12 5633: positiveSegments: 16, negativeSegments: 0 5635: positiveSegments: 7, negativeSegments: 0 5637: positiveSegments: 11, negativeSegments: 15 5638: positiveSegments: 18, negativeSegments: 3 5641: positiveSegments: 23, negativeSegments: 6 5642: positiveSegments: 11, negativeSegments: 0 5646: positiveSegments: 0, negativeSegments: 3 5647: positiveSegments: 4, negativeSegments: 3 5648: positiveSegments: 13, negativeSegments: 6 5650: positiveSegments: 10, negativeSegments: 3 5654: positiveSegments: 0, negativeSegments: 6 5655: positiveSegments: 14, negativeSegments: 3 5657: positiveSegments: 9, negativeSegments: 3 5658: positiveSegments: 15, negativeSegments: 3 5659: positiveSegments: 0, negativeSegments: 12 5662: positiveSegments: 2, negativeSegments: 0 5664: exit early, no segments to save 5664: positiveSegments: 0, negativeSegments: 0 5665: positiveSegments: 8, negativeSegments: 12 5669: positiveSegments: 8, negativeSegments: 9 5670: positiveSegments: 13, negativeSegments: 9 5671: positiveSegments: 0, negativeSegments: 6 5673: positiveSegments: 19, negativeSegments: 3 5675: positiveSegments: 6, negativeSegments: 9 5677: positiveSegments: 4, negativeSegments: 6 5678: positiveSegments: 19, negativeSegments: 30 5680: positiveSegments: 4, negativeSegments: 6 5682: positiveSegments: 24, negativeSegments: 0 5684: positiveSegments: 9, negativeSegments: 6 5685: positiveSegments: 9, negativeSegments: 6 5687: positiveSegments: 33, negativeSegments: 6 5690: positiveSegments: 7, negativeSegments: 6 5691: positiveSegments: 5, negativeSegments: 0 5692: positiveSegments: 0, negativeSegments: 6 5693: exit early, no segments to save 5693: positiveSegments: 0, negativeSegments: 0 5694: positiveSegments: 6, negativeSegments: 6 5696: positiveSegments: 10, negativeSegments: 0 5698: positiveSegments: 41, negativeSegments: 0 5703: positiveSegments: 8, negativeSegments: 6 5711: exit early, no segments to save 5711: positiveSegments: 0, negativeSegments: 0 5715: positiveSegments: 16, negativeSegments: 18 5717: positiveSegments: 4, negativeSegments: 0 5718: positiveSegments: 4, negativeSegments: 3 5719: positiveSegments: 5, negativeSegments: 3 5721: positiveSegments: 1, negativeSegments: 6 5724: positiveSegments: 0, negativeSegments: 21 5725: positiveSegments: 10, negativeSegments: 9 5727: positiveSegments: 0, negativeSegments: 6 5729: positiveSegments: 2, negativeSegments: 15 5733: positiveSegments: 4, negativeSegments: 15 5734: positiveSegments: 3, negativeSegments: 0 5743: positiveSegments: 8, negativeSegments: 3 5745: positiveSegments: 7, negativeSegments: 0 5746: positiveSegments: 32, negativeSegments: 3 5749: positiveSegments: 10, negativeSegments: 6 5750: positiveSegments: 14, negativeSegments: 0 5751: positiveSegments: 8, negativeSegments: 12 5753: positiveSegments: 6, negativeSegments: 0 5755: positiveSegments: 12, negativeSegments: 0 5759: positiveSegments: 5, negativeSegments: 6 5760: positiveSegments: 27, negativeSegments: 18 5765: positiveSegments: 7, negativeSegments: 0 5769: positiveSegments: 2, negativeSegments: 12 count processed: 2800, current case index: 5771 5771: positiveSegments: 24, negativeSegments: 3 5772: positiveSegments: 9, negativeSegments: 12 5777: positiveSegments: 0, negativeSegments: 9 5780: positiveSegments: 15, negativeSegments: 6 5781: positiveSegments: 12, negativeSegments: 3 5782: positiveSegments: 2, negativeSegments: 3 5783: positiveSegments: 11, negativeSegments: 15 5784: positiveSegments: 11, negativeSegments: 0 5787: positiveSegments: 58, negativeSegments: 0 5788: positiveSegments: 8, negativeSegments: 12 5793: positiveSegments: 0, negativeSegments: 9 5795: positiveSegments: 19, negativeSegments: 6 5799: positiveSegments: 8, negativeSegments: 0 5800: exit early, no segments to save 5800: positiveSegments: 0, negativeSegments: 0 5801: positiveSegments: 0, negativeSegments: 3 5805: positiveSegments: 6, negativeSegments: 0 5806: positiveSegments: 32, negativeSegments: 3 5808: positiveSegments: 7, negativeSegments: 3 5809: positiveSegments: 5, negativeSegments: 21 5810: exit early, no segments to save 5810: positiveSegments: 0, negativeSegments: 0 5811: positiveSegments: 4, negativeSegments: 0 5814: positiveSegments: 8, negativeSegments: 0 5816: positiveSegments: 3, negativeSegments: 9 5817: exit early, no segments to save 5817: positiveSegments: 0, negativeSegments: 0 5819: positiveSegments: 12, negativeSegments: 15 5823: exit early, no segments to save 5823: positiveSegments: 0, negativeSegments: 0 5825: positiveSegments: 11, negativeSegments: 12 5826: positiveSegments: 17, negativeSegments: 15 5827: positiveSegments: 16, negativeSegments: 9 5829: positiveSegments: 8, negativeSegments: 6 5831: positiveSegments: 29, negativeSegments: 3 5832: positiveSegments: 8, negativeSegments: 3 5834: positiveSegments: 0, negativeSegments: 3 5837: positiveSegments: 20, negativeSegments: 6 5839: positiveSegments: 6, negativeSegments: 9 5840: positiveSegments: 4, negativeSegments: 21 5842: positiveSegments: 0, negativeSegments: 9 5843: positiveSegments: 8, negativeSegments: 12 5844: positiveSegments: 14, negativeSegments: 6 5848: positiveSegments: 7, negativeSegments: 3 5849: positiveSegments: 19, negativeSegments: 0 5851: positiveSegments: 0, negativeSegments: 9 5859: positiveSegments: 21, negativeSegments: 9 5860: positiveSegments: 12, negativeSegments: 9 5861: positiveSegments: 7, negativeSegments: 0 5862: positiveSegments: 68, negativeSegments: 0 5864: positiveSegments: 7, negativeSegments: 3 5865: positiveSegments: 19, negativeSegments: 3 5866: positiveSegments: 20, negativeSegments: 9 5869: positiveSegments: 14, negativeSegments: 9 5870: positiveSegments: 5, negativeSegments: 9 5871: exit early, no segments to save 5871: positiveSegments: 0, negativeSegments: 0 5872: positiveSegments: 24, negativeSegments: 6 5873: positiveSegments: 8, negativeSegments: 3 5875: positiveSegments: 32, negativeSegments: 3 5882: exit early, no segments to save 5882: positiveSegments: 0, negativeSegments: 0 5884: positiveSegments: 8, negativeSegments: 0 5887: positiveSegments: 8, negativeSegments: 0 5888: positiveSegments: 4, negativeSegments: 3 5889: positiveSegments: 4, negativeSegments: 3 5890: positiveSegments: 12, negativeSegments: 3 5891: positiveSegments: 0, negativeSegments: 3 5892: positiveSegments: 4, negativeSegments: 3 5894: positiveSegments: 15, negativeSegments: 0 5895: positiveSegments: 5, negativeSegments: 9 5902: positiveSegments: 7, negativeSegments: 3 5904: positiveSegments: 16, negativeSegments: 6 5907: positiveSegments: 6, negativeSegments: 3 5908: exit early, no segments to save 5908: positiveSegments: 0, negativeSegments: 0 5911: positiveSegments: 8, negativeSegments: 3 5912: positiveSegments: 4, negativeSegments: 6 5914: positiveSegments: 12, negativeSegments: 3 5916: exit early, no segments to save 5916: positiveSegments: 0, negativeSegments: 0 5917: positiveSegments: 10, negativeSegments: 0 5918: positiveSegments: 4, negativeSegments: 9 5933: positiveSegments: 9, negativeSegments: 0 5934: positiveSegments: 1, negativeSegments: 6 5937: positiveSegments: 15, negativeSegments: 0 5938: positiveSegments: 4, negativeSegments: 6 5940: positiveSegments: 4, negativeSegments: 0 5942: positiveSegments: 11, negativeSegments: 3 5943: positiveSegments: 4, negativeSegments: 3 5944: positiveSegments: 12, negativeSegments: 9 5945: positiveSegments: 8, negativeSegments: 0 5946: positiveSegments: 12, negativeSegments: 6 5948: positiveSegments: 9, negativeSegments: 0 5950: positiveSegments: 29, negativeSegments: 9 5951: positiveSegments: 0, negativeSegments: 3 5954: positiveSegments: 7, negativeSegments: 0 5956: positiveSegments: 8, negativeSegments: 6 5958: positiveSegments: 0, negativeSegments: 15 5959: exit early, no segments to save 5959: positiveSegments: 0, negativeSegments: 0 5961: exit early, no segments to save 5961: positiveSegments: 0, negativeSegments: 0 5964: positiveSegments: 0, negativeSegments: 18 5965: positiveSegments: 19, negativeSegments: 0 5966: positiveSegments: 8, negativeSegments: 6 5967: positiveSegments: 21, negativeSegments: 9 5970: positiveSegments: 4, negativeSegments: 3 5971: nothing saved, all segments filtered 5971: positiveSegments: 1, negativeSegments: 0 5973: positiveSegments: 2, negativeSegments: 6 count processed: 2900, current case index: 5974 5974: exit early, no segments to save 5974: positiveSegments: 0, negativeSegments: 0 5975: positiveSegments: 12, negativeSegments: 3 5976: positiveSegments: 16, negativeSegments: 0 5977: positiveSegments: 15, negativeSegments: 9 5981: positiveSegments: 19, negativeSegments: 0 5982: positiveSegments: 6, negativeSegments: 0 5983: positiveSegments: 19, negativeSegments: 0 5986: positiveSegments: 8, negativeSegments: 9 5987: positiveSegments: 4, negativeSegments: 3 5989: positiveSegments: 13, negativeSegments: 3 5993: positiveSegments: 24, negativeSegments: 9 5994: positiveSegments: 7, negativeSegments: 0 5997: positiveSegments: 10, negativeSegments: 6 6000: positiveSegments: 4, negativeSegments: 21 6003: positiveSegments: 12, negativeSegments: 0 6006: nothing saved, all segments filtered 6006: positiveSegments: 3, negativeSegments: 0 6007: positiveSegments: 17, negativeSegments: 3 6009: positiveSegments: 36, negativeSegments: 6 6010: positiveSegments: 4, negativeSegments: 12 6013: positiveSegments: 8, negativeSegments: 0 6015: positiveSegments: 8, negativeSegments: 0 6016: positiveSegments: 0, negativeSegments: 3 6017: positiveSegments: 1, negativeSegments: 9 6020: positiveSegments: 5, negativeSegments: 15 6022: positiveSegments: 16, negativeSegments: 0 6027: positiveSegments: 4, negativeSegments: 0 6028: positiveSegments: 11, negativeSegments: 0 6029: positiveSegments: 2, negativeSegments: 3 6031: positiveSegments: 8, negativeSegments: 3 6032: exit early, no segments to save 6032: positiveSegments: 0, negativeSegments: 0 6037: positiveSegments: 12, negativeSegments: 6 6039: positiveSegments: 17, negativeSegments: 12 6041: exit early, no segments to save 6041: positiveSegments: 0, negativeSegments: 0 6042: positiveSegments: 7, negativeSegments: 0 6043: positiveSegments: 15, negativeSegments: 6 6047: positiveSegments: 3, negativeSegments: 9 6053: positiveSegments: 1, negativeSegments: 6 6055: positiveSegments: 6, negativeSegments: 0 6056: positiveSegments: 3, negativeSegments: 0 6057: exit early, no segments to save 6057: positiveSegments: 0, negativeSegments: 0 6058: positiveSegments: 32, negativeSegments: 9 6059: positiveSegments: 0, negativeSegments: 12 6060: positiveSegments: 0, negativeSegments: 12 6061: positiveSegments: 11, negativeSegments: 15 6063: positiveSegments: 11, negativeSegments: 9 6065: positiveSegments: 3, negativeSegments: 3 6066: positiveSegments: 18, negativeSegments: 0 6067: exit early, no segments to save 6067: positiveSegments: 0, negativeSegments: 0 6069: positiveSegments: 30, negativeSegments: 12 6070: positiveSegments: 0, negativeSegments: 6 6071: positiveSegments: 26, negativeSegments: 0 6074: positiveSegments: 4, negativeSegments: 18 6076: positiveSegments: 11, negativeSegments: 6 6077: positiveSegments: 17, negativeSegments: 6 6080: positiveSegments: 6, negativeSegments: 0 6082: positiveSegments: 29, negativeSegments: 3 6083: positiveSegments: 13, negativeSegments: 9 6084: positiveSegments: 44, negativeSegments: 18 6085: positiveSegments: 20, negativeSegments: 3 6086: positiveSegments: 21, negativeSegments: 6 6087: positiveSegments: 8, negativeSegments: 3 6088: exit early, no segments to save 6088: positiveSegments: 0, negativeSegments: 0 6089: positiveSegments: 0, negativeSegments: 6 6097: exit early, no segments to save 6097: positiveSegments: 0, negativeSegments: 0 6098: positiveSegments: 5, negativeSegments: 3 6101: positiveSegments: 6, negativeSegments: 6 6102: positiveSegments: 9, negativeSegments: 3 6103: positiveSegments: 0, negativeSegments: 12 6104: positiveSegments: 4, negativeSegments: 6 6109: exit early, no segments to save 6109: positiveSegments: 0, negativeSegments: 0 6114: positiveSegments: 26, negativeSegments: 0 6119: positiveSegments: 8, negativeSegments: 12 6121: positiveSegments: 25, negativeSegments: 3 6124: positiveSegments: 16, negativeSegments: 0 6126: positiveSegments: 2, negativeSegments: 18 6127: positiveSegments: 11, negativeSegments: 0 6129: positiveSegments: 14, negativeSegments: 12 6131: nothing saved, all segments filtered 6131: positiveSegments: 2, negativeSegments: 0 6132: positiveSegments: 13, negativeSegments: 0 6133: positiveSegments: 6, negativeSegments: 9 6134: positiveSegments: 3, negativeSegments: 9 6135: positiveSegments: 5, negativeSegments: 3 6136: positiveSegments: 8, negativeSegments: 6 6140: positiveSegments: 2, negativeSegments: 3 6141: positiveSegments: 15, negativeSegments: 0 6143: positiveSegments: 8, negativeSegments: 9 6144: positiveSegments: 7, negativeSegments: 0 6147: positiveSegments: 0, negativeSegments: 15 6152: positiveSegments: 15, negativeSegments: 0 6153: positiveSegments: 12, negativeSegments: 9 6154: positiveSegments: 4, negativeSegments: 0 6156: positiveSegments: 0, negativeSegments: 6 6159: positiveSegments: 43, negativeSegments: 3 6160: positiveSegments: 31, negativeSegments: 0 6163: positiveSegments: 16, negativeSegments: 3 6166: positiveSegments: 42, negativeSegments: 3 6167: positiveSegments: 0, negativeSegments: 15 6168: positiveSegments: 0, negativeSegments: 3 6169: positiveSegments: 0, negativeSegments: 12 6171: positiveSegments: 15, negativeSegments: 6 count processed: 3000, current case index: 6174 6174: positiveSegments: 0, negativeSegments: 6 6176: positiveSegments: 23, negativeSegments: 9 6178: positiveSegments: 23, negativeSegments: 3 6179: positiveSegments: 35, negativeSegments: 9 6180: nothing saved, all segments filtered 6180: positiveSegments: 3, negativeSegments: 0 6182: positiveSegments: 15, negativeSegments: 6 6184: positiveSegments: 3, negativeSegments: 12 6185: positiveSegments: 4, negativeSegments: 6 6186: positiveSegments: 0, negativeSegments: 6 6190: positiveSegments: 9, negativeSegments: 9 6191: positiveSegments: 8, negativeSegments: 3 6192: exit early, no segments to save 6192: positiveSegments: 0, negativeSegments: 0 6194: positiveSegments: 2, negativeSegments: 3 6195: positiveSegments: 16, negativeSegments: 3 6196: positiveSegments: 0, negativeSegments: 9 6198: positiveSegments: 3, negativeSegments: 6 6199: positiveSegments: 7, negativeSegments: 6 6200: positiveSegments: 7, negativeSegments: 3 6204: positiveSegments: 1, negativeSegments: 0 6205: exit early, no segments to save 6205: positiveSegments: 0, negativeSegments: 0 6206: positiveSegments: 6, negativeSegments: 0 6208: positiveSegments: 0, negativeSegments: 3 6210: positiveSegments: 1, negativeSegments: 3 6214: positiveSegments: 13, negativeSegments: 6 6217: positiveSegments: 37, negativeSegments: 3 6218: positiveSegments: 7, negativeSegments: 6 6219: positiveSegments: 7, negativeSegments: 9 6220: positiveSegments: 30, negativeSegments: 12 6224: positiveSegments: 4, negativeSegments: 6 6227: positiveSegments: 21, negativeSegments: 0 6228: positiveSegments: 9, negativeSegments: 0 6230: positiveSegments: 8, negativeSegments: 3 6233: positiveSegments: 6, negativeSegments: 0 6235: positiveSegments: 0, negativeSegments: 3 6238: positiveSegments: 14, negativeSegments: 3 6239: positiveSegments: 19, negativeSegments: 6 6240: positiveSegments: 4, negativeSegments: 0 6241: positiveSegments: 7, negativeSegments: 9 6248: positiveSegments: 19, negativeSegments: 6 6250: positiveSegments: 4, negativeSegments: 6 6254: positiveSegments: 5, negativeSegments: 6 6255: positiveSegments: 11, negativeSegments: 12 6257: positiveSegments: 5, negativeSegments: 24 6260: positiveSegments: 10, negativeSegments: 0 6261: exit early, no segments to save 6261: positiveSegments: 0, negativeSegments: 0 6262: positiveSegments: 5, negativeSegments: 3 6264: positiveSegments: 0, negativeSegments: 15 6266: positiveSegments: 21, negativeSegments: 3 6267: positiveSegments: 4, negativeSegments: 6 6268: positiveSegments: 4, negativeSegments: 3 6269: positiveSegments: 13, negativeSegments: 0 6270: positiveSegments: 5, negativeSegments: 18 6271: positiveSegments: 1, negativeSegments: 0 6273: positiveSegments: 11, negativeSegments: 6 6275: positiveSegments: 4, negativeSegments: 0 6277: positiveSegments: 3, negativeSegments: 0 6279: positiveSegments: 5, negativeSegments: 12 6280: positiveSegments: 2, negativeSegments: 9 6281: positiveSegments: 26, negativeSegments: 0 6282: positiveSegments: 16, negativeSegments: 6 6284: positiveSegments: 9, negativeSegments: 3 6286: positiveSegments: 1, negativeSegments: 3 6289: positiveSegments: 6, negativeSegments: 3 6290: positiveSegments: 21, negativeSegments: 0 6292: positiveSegments: 10, negativeSegments: 3 6293: exit early, no segments to save 6293: positiveSegments: 0, negativeSegments: 0 6295: positiveSegments: 8, negativeSegments: 6 6296: positiveSegments: 19, negativeSegments: 6 6297: positiveSegments: 35, negativeSegments: 0 6298: positiveSegments: 5, negativeSegments: 6 6302: positiveSegments: 10, negativeSegments: 0 6305: positiveSegments: 3, negativeSegments: 12 6306: positiveSegments: 6, negativeSegments: 3 6307: positiveSegments: 12, negativeSegments: 3 6309: positiveSegments: 68, negativeSegments: 3 6311: positiveSegments: 16, negativeSegments: 3 6312: positiveSegments: 14, negativeSegments: 0 6314: positiveSegments: 18, negativeSegments: 3 6315: positiveSegments: 4, negativeSegments: 0 6316: positiveSegments: 17, negativeSegments: 3 6317: positiveSegments: 23, negativeSegments: 0 6324: positiveSegments: 5, negativeSegments: 0 6330: positiveSegments: 4, negativeSegments: 0 6331: positiveSegments: 3, negativeSegments: 0 6332: positiveSegments: 8, negativeSegments: 15 6339: positiveSegments: 4, negativeSegments: 0 6343: positiveSegments: 8, negativeSegments: 12 6345: positiveSegments: 8, negativeSegments: 9 6346: positiveSegments: 0, negativeSegments: 6 6351: positiveSegments: 20, negativeSegments: 9 6355: positiveSegments: 3, negativeSegments: 6 6357: positiveSegments: 8, negativeSegments: 12 6359: positiveSegments: 7, negativeSegments: 0 6360: positiveSegments: 30, negativeSegments: 0 6361: positiveSegments: 26, negativeSegments: 3 6362: positiveSegments: 15, negativeSegments: 3 6363: positiveSegments: 12, negativeSegments: 12 6366: positiveSegments: 2, negativeSegments: 6 6368: positiveSegments: 8, negativeSegments: 0 6370: positiveSegments: 0, negativeSegments: 6 count processed: 3100, current case index: 6372 6372: positiveSegments: 0, negativeSegments: 9 6373: positiveSegments: 2, negativeSegments: 0 6375: positiveSegments: 10, negativeSegments: 0 6376: positiveSegments: 12, negativeSegments: 0 6378: positiveSegments: 18, negativeSegments: 0 6381: positiveSegments: 0, negativeSegments: 3 6383: positiveSegments: 4, negativeSegments: 15 6385: positiveSegments: 29, negativeSegments: 0 6386: positiveSegments: 8, negativeSegments: 15 6388: positiveSegments: 9, negativeSegments: 0 extracted: 3110
def printAbp(case_id_to_check, plot_invalid_only=False):
vf_path = f'{VITAL_MINI}/{case_id_to_check:04d}_mini.vital'
if not os.path.isfile(vf_path):
return
vf = vitaldb.VitalFile(vf_path)
abp = vf.to_numpy(TRACK_NAMES[0], 1/500)
print(f'Case {case_id_to_check}')
print(f'ABP Shape: {abp.shape}')
print(f'nanmin: {np.nanmin(abp)}')
print(f'nanmean: {np.nanmean(abp)}')
print(f'nanmax: {np.nanmax(abp)}')
is_valid = isAbpSegmentValidNumpy(abp, debug=True)
print(f'valid: {is_valid}')
if plot_invalid_only and is_valid:
return
plt.figure(figsize=(20, 5))
plt_color = 'C0' if is_valid else 'red'
plt.plot(abp, plt_color)
plt.title(f'ABP - Entire Track - Case {case_id_to_check} - {abp.shape[0] / 500} seconds')
plt.axhline(y = 65, color = 'maroon', linestyle = '--')
plt.show()
def printSegments(segmentsMap, case_id_to_check, print_label, normalize=False):
for (x1, x2, r, abp, ecg, eeg) in segmentsMap[case_id_to_check]:
print(f'{print_label}: Case {case_id_to_check}')
print(f'lookback window: {r} min')
print(f'start time: {x1}')
print(f'end time: {x2}')
print(f'length: {x2 - x1} sec')
print(f'ABP Shape: {abp.shape}')
print(f'ECG Shape: {ecg.shape}')
print(f'EEG Shape: {eeg.shape}')
print(f'nanmin: {np.nanmin(abp)}')
print(f'nanmean: {np.nanmean(abp)}')
print(f'nanmax: {np.nanmax(abp)}')
is_valid = isAbpSegmentValidNumpy(abp, debug=True)
print(f'valid: {is_valid}')
# ABP normalization
x_abp = np.copy(abp)
if normalize:
x_abp -= 65
x_abp /= 65
plt.figure(figsize=(20, 5))
plt_color = 'C0' if is_valid else 'red'
plt.plot(x_abp, plt_color)
plt.title('ABP')
plt.axhline(y = 65, color = 'maroon', linestyle = '--')
plt.show()
plt.figure(figsize=(20, 5))
plt.plot(ecg, 'teal')
plt.title('ECG')
plt.show()
plt.figure(figsize=(20, 5))
plt.plot(eeg, 'indigo')
plt.title('EEG')
plt.show()
print()
def printEvents(abp_raw, eventsMap, case_id_to_check, print_label, normalize=False):
for (x1, x2) in eventsMap[case_id_to_check]:
print(f'{print_label}: Case {case_id_to_check}')
print(f'start time: {x1}')
print(f'end time: {x2}')
print(f'length: {x2 - x1} sec')
abp = abp_raw[x1*500:x2*500]
print(f'ABP Shape: {abp.shape}')
print(f'nanmin: {np.nanmin(abp)}')
print(f'nanmean: {np.nanmean(abp)}')
print(f'nanmax: {np.nanmax(abp)}')
is_valid = isAbpSegmentValidNumpy(abp, debug=True)
print(f'valid: {is_valid}')
# ABP normalization
x_abp = np.copy(abp)
if normalize:
x_abp -= 65
x_abp /= 65
plt.figure(figsize=(20, 5))
plt_color = 'C0' if is_valid else 'red'
plt.plot(x_abp, plt_color)
plt.title('ABP')
plt.axhline(y = 65, color = 'maroon', linestyle = '--')
plt.show()
print()
def moving_average(x, seconds=60):
w = seconds * 500
return np.convolve(np.squeeze(x), np.ones(w), 'valid') / w
def printAbpOverlay(
case_id_to_check,
positiveSegmentsMap,
negativeSegmentsMap,
iohEventsMap,
cleanEventsMap,
movingAverage=False
):
def overlay_segments(plt, segmentsMap, color, linestyle, positive=False):
for (x1, x2, r, abp, ecg, eeg) in segmentsMap:
sx1 = x1*500
sx2 = x2*500
mycolor = color
if positive:
if r == 3:
mycolor = 'red'
elif r == 5:
mycolor = 'crimson'
elif r == 10:
mycolor = 'tomato'
else:
mycolor = 'salmon'
plt.axvline(x = sx1, color = mycolor, linestyle = linestyle)
plt.axvline(x = sx2, color = mycolor, linestyle = linestyle)
plt.axvspan(sx1, sx2, facecolor = mycolor, alpha = 0.1)
def overlay_events(plt, eventsMap, color, linestyle):
for (x1, x2) in eventsMap:
sx1 = x1*500
sx2 = x2*500
plt.axvline(x = sx1, color = color, linestyle = linestyle)
plt.axvline(x = sx2, color = color, linestyle = linestyle)
plt.axvspan(sx1, sx2, facecolor = color, alpha = 0.1)
vf_path = f'{VITAL_MINI}/{case_id_to_check:04d}_mini.vital'
if not os.path.isfile(vf_path):
return
vf = vitaldb.VitalFile(vf_path)
abp = vf.to_numpy(TRACK_NAMES[0], 1/500)
abp_mov_avg = None
if movingAverage:
abp_mov_avg = moving_average(abp)
print(f'Case {case_id_to_check}')
print(f'ABP Shape: {abp.shape}')
print(f'nanmin: {np.nanmin(abp)}')
print(f'nanmean: {np.nanmean(abp)}')
print(f'nanmax: {np.nanmax(abp)}')
is_valid = isAbpSegmentValidNumpy(abp, debug=True)
print(f'valid: {is_valid}')
plt.figure(figsize=(24, 8))
plt_color = 'C0' if is_valid else 'red'
plt.plot(abp, plt_color)
plt.title(f'ABP - Entire Track - Case {case_id_to_check} - {abp.shape[0] / 500} seconds')
plt.axhline(y = 65, color = 'maroon', linestyle = '--')
if movingAverage:
plt.plot(abp_mov_avg, 'maroon')
# https://matplotlib.org/stable/gallery/lines_bars_and_markers/linestyles.html#linestyles
overlay_segments(plt, positiveSegmentsMap[case_id_to_check], 'crimson', (0, (1, 1)), positive=True)
overlay_segments(plt, negativeSegmentsMap[case_id_to_check], 'teal', (0, (1, 1)))
overlay_events(plt, iohEventsMap[case_id_to_check], 'brown', '-')
overlay_events(plt, cleanEventsMap[case_id_to_check], 'teal', '-')
plt.show()
# Check if all ABPs are well formed.
DISPLAY_REALITY_CHECK_ABP=True
DISPLAY_REALITY_CHECK_ABP_FIRST_ONLY=True
if DISPLAY_REALITY_CHECK_ABP:
for case_id_to_check in cases_of_interest_idx:
printAbp(case_id_to_check, plot_invalid_only=False)
if DISPLAY_REALITY_CHECK_ABP_FIRST_ONLY:
break
Case 1 ABP Shape: (5770575, 1) nanmin: -495.6260070800781 nanmean: 78.15251159667969 nanmax: 374.3236389160156 Presence of BP > 200 valid: False
# These are Vital Files removed because of malformed ABP waveforms.
DISPLAY_MALFORMED_ABP=True
DISPLAY_MALFORMED_ABP_FIRST_ONLY=True
if DISPLAY_MALFORMED_ABP:
malformed_case_ids = pd.read_csv('malformed_tracks_filter.csv', header=None, names=['caseid']).set_index('caseid').index
for case_id_to_check in malformed_case_ids:
printAbp(case_id_to_check)
if DISPLAY_MALFORMED_ABP_FIRST_ONLY:
break
Case 3 ABP Shape: (2196524, 1) nanmin: -117.43000030517578 nanmean: 0.6060270667076111 nanmax: 85.98619842529297 Presence of BP < 30 valid: False
DISPLAY_NO_SEGMENTS_CASES=True
DISPLAY_NO_SEGMENTS_CASES_FIRST_ONLY=True
if DISPLAY_NO_SEGMENTS_CASES:
no_segments_case_ids = [3413, 3476, 3533, 3992, 4328, 4648, 4703, 4733, 5130, 5501, 5693, 5908]
for case_id_to_check in no_segments_case_ids:
printAbp(case_id_to_check)
if DISPLAY_NO_SEGMENTS_CASES_FIRST_ONLY:
break
Case 3413 ABP Shape: (3429927, 1) nanmin: -228.025146484375 nanmean: 48.4425163269043 nanmax: 293.3521423339844 >10% NaN valid: False
Generate segment data for one or more cases. Perform a deep analysis of event and segment quality.
#mycoi = cases_of_interest_idx
my_cases_of_interest_idx = cases_of_interest_idx[:10]
#mycoi = [1]
positiveSegmentsMap, negativeSegmentsMap, iohEventsMap, cleanEventsMap = \
extract_segments(my_cases_of_interest_idx, debug=False, checkCache=False,
forceWrite=False, returnSegments=True, skipInvalidCleanEvents=True)
1: positiveSegments: 12, negativeSegments: 3 4: positiveSegments: 22, negativeSegments: 3 7: positiveSegments: 12, negativeSegments: 6 10: positiveSegments: 27, negativeSegments: 6 12: positiveSegments: 22, negativeSegments: 0 13: positiveSegments: 18, negativeSegments: 0 16: positiveSegments: 12, negativeSegments: 6 19: positiveSegments: 34, negativeSegments: 3 20: positiveSegments: 17, negativeSegments: 9 22: positiveSegments: 16, negativeSegments: 12
Select a specific case to perform detailed low level analysis.
case_id_to_check = my_cases_of_interest_idx[0]
case_id_to_check
1
print((
len(positiveSegmentsMap[case_id_to_check]),
len(negativeSegmentsMap[case_id_to_check]),
len(iohEventsMap[case_id_to_check]),
len(cleanEventsMap[case_id_to_check])
))
(12, 3, 7, 3)
printAbp(case_id_to_check)
Case 1 ABP Shape: (5770575, 1) nanmin: -495.6260070800781 nanmean: 78.15251159667969 nanmax: 374.3236389160156 Presence of BP > 200 valid: False
Used to define the range in front of which positive segments will be extracted. Positive samples happen in front of this region.
tmp_vf_path = f'{VITAL_MINI}/{case_id_to_check:04d}_mini.vital'
tmp_vf = vitaldb.VitalFile(tmp_vf_path)
tmp_abp = tmp_vf.to_numpy(TRACK_NAMES[0], 1/500)
printEvents(tmp_abp, iohEventsMap, case_id_to_check, 'IOH Event Segment', normalize=False)
IOH Event Segment: Case 1 start time: 1788 end time: 1849 length: 61 sec ABP Shape: (30500, 1) nanmin: 32.663482666015625 nanmean: 64.93988037109375 nanmax: 123.50955200195312 valid: True
IOH Event Segment: Case 1 start time: 1850 end time: 2113 length: 263 sec ABP Shape: (131500, 1) nanmin: 37.600799560546875 nanmean: 63.139060974121094 nanmax: 101.78549194335938 valid: True
IOH Event Segment: Case 1 start time: 2314 end time: 2375 length: 61 sec ABP Shape: (30500, 1) nanmin: -262.5861511230469 nanmean: 65.14369201660156 nanmax: 343.7124938964844 Presence of BP > 200 valid: False
IOH Event Segment: Case 1 start time: 4113 end time: 4199 length: 86 sec ABP Shape: (43000, 1) nanmin: 22.788909912109375 nanmean: 65.0725326538086 nanmax: 153.13327026367188 Presence of BP < 30 valid: False
IOH Event Segment: Case 1 start time: 4261 end time: 5350 length: 1089 sec ABP Shape: (544500, 1) nanmin: 36.613311767578125 nanmean: 60.451026916503906 nanmax: 110.67263793945312 valid: True
IOH Event Segment: Case 1 start time: 9096 end time: 9156 length: 60 sec ABP Shape: (30000, 1) nanmin: 40.563140869140625 nanmean: 64.9837646484375 nanmax: 108.69772338867188 valid: True
IOH Event Segment: Case 1 start time: 9157 end time: 9503 length: 346 sec ABP Shape: (173000, 1) nanmin: 39.575714111328125 nanmean: 62.33021545410156 nanmax: 104.74789428710938 valid: True
Used to define the range from in which negative segments will be extracted. Negative samples happen within this region.
printEvents(tmp_abp, cleanEventsMap, case_id_to_check, 'Clean Event Segment', normalize=False)
Clean Event Segment: Case 1 start time: 5351 end time: 7151 length: 1800 sec ABP Shape: (900000, 1) nanmin: 40.563140869140625 nanmean: 84.04818725585938 nanmax: 151.15835571289062 valid: True
Clean Event Segment: Case 1 start time: 7152 end time: 8952 length: 1800 sec ABP Shape: (900000, 1) nanmin: -495.6260070800781 nanmean: 99.71124267578125 nanmax: 368.3988952636719 Presence of BP > 200 valid: False
Clean Event Segment: Case 1 start time: 9504 end time: 11304 length: 1800 sec ABP Shape: (900000, 1) nanmin: -49.295440673828125 nanmean: 83.3201675415039 nanmax: 346.6748352050781 Presence of BP > 200 valid: False
One minute regions sampled and used for training the model for "positive" events.
printSegments(positiveSegmentsMap, case_id_to_check, 'Positive Segment - IOH Event', normalize=False)
Positive Segment - IOH Event: Case 1 lookback window: 3 min start time: 1548 end time: 1608 length: 60 sec ABP Shape: (30000,) ECG Shape: (30000,) EEG Shape: (7680,) nanmin: 46.487884521484375 nanmean: 73.00869750976562 nanmax: 113.63497924804688 valid: True
Positive Segment - IOH Event: Case 1 lookback window: 5 min start time: 1428 end time: 1488 length: 60 sec ABP Shape: (30000,) ECG Shape: (30000,) EEG Shape: (7680,) nanmin: 41.550628662109375 nanmean: 74.47395324707031 nanmax: 128.44686889648438 valid: True
Positive Segment - IOH Event: Case 1 lookback window: 10 min start time: 1128 end time: 1188 length: 60 sec ABP Shape: (30000,) ECG Shape: (30000,) EEG Shape: (7680,) nanmin: 53.400115966796875 nanmean: 88.63211059570312 nanmax: 135.35903930664062 valid: True
Positive Segment - IOH Event: Case 1 lookback window: 15 min start time: 828 end time: 888 length: 60 sec ABP Shape: (30000,) ECG Shape: (30000,) EEG Shape: (7680,) nanmin: 23.776397705078125 nanmean: 108.88127136230469 nanmax: 182.75698852539062 Presence of BP < 30 valid: False
Positive Segment - IOH Event: Case 1 lookback window: 3 min start time: 3873 end time: 3933 length: 60 sec ABP Shape: (30000,) ECG Shape: (30000,) EEG Shape: (7680,) nanmin: 46.487884521484375 nanmean: 75.3544692993164 nanmax: 124.49703979492188 valid: True
Positive Segment - IOH Event: Case 1 lookback window: 5 min start time: 3753 end time: 3813 length: 60 sec ABP Shape: (30000,) ECG Shape: (30000,) EEG Shape: (7680,) nanmin: 45.500457763671875 nanmean: 73.97709655761719 nanmax: 122.52212524414062 valid: True
Positive Segment - IOH Event: Case 1 lookback window: 10 min start time: 3453 end time: 3513 length: 60 sec ABP Shape: (30000,) ECG Shape: (30000,) EEG Shape: (7680,) nanmin: 52.412628173828125 nanmean: 86.52787780761719 nanmax: 148.19595336914062 valid: True
Positive Segment - IOH Event: Case 1 lookback window: 15 min start time: 3153 end time: 3213 length: 60 sec ABP Shape: (30000,) ECG Shape: (30000,) EEG Shape: (7680,) nanmin: 58.337371826171875 nanmean: 100.94121551513672 nanmax: 165.97018432617188 valid: True
Positive Segment - IOH Event: Case 1 lookback window: 3 min start time: 8856 end time: 8916 length: 60 sec ABP Shape: (30000,) ECG Shape: (30000,) EEG Shape: (7680,) nanmin: 64.26211547851562 nanmean: 97.06536102294922 nanmax: 157.08309936523438 valid: True
Positive Segment - IOH Event: Case 1 lookback window: 5 min start time: 8736 end time: 8796 length: 60 sec ABP Shape: (30000,) ECG Shape: (30000,) EEG Shape: (7680,) nanmin: 69.19943237304688 nanmean: 105.55238342285156 nanmax: 163.00784301757812 valid: True
Positive Segment - IOH Event: Case 1 lookback window: 10 min start time: 8436 end time: 8496 length: 60 sec ABP Shape: (30000,) ECG Shape: (30000,) EEG Shape: (7680,) nanmin: -88.793701171875 nanmean: 130.62982177734375 nanmax: 305.2016296386719 Presence of BP > 200 valid: False
Positive Segment - IOH Event: Case 1 lookback window: 15 min start time: 8136 end time: 8196 length: 60 sec ABP Shape: (30000,) ECG Shape: (30000,) EEG Shape: (7680,) nanmin: 62.287200927734375 nanmean: 92.04357147216797 nanmax: 138.32138061523438 valid: True
One minute regions sampled and used for training the model for "negative" events.
printSegments(negativeSegmentsMap, case_id_to_check, 'Negative Segment - Non-Event', normalize=False)
Negative Segment - Non-Event: Case 1 lookback window: 0 min start time: 5951 end time: 6011 length: 60 sec ABP Shape: (30000,) ECG Shape: (30000,) EEG Shape: (7680,) nanmin: 52.412628173828125 nanmean: 76.35643005371094 nanmax: 120.54721069335938 valid: True
Negative Segment - Non-Event: Case 1 lookback window: 0 min start time: 6251 end time: 6311 length: 60 sec ABP Shape: (30000,) ECG Shape: (30000,) EEG Shape: (7680,) nanmin: 54.387542724609375 nanmean: 77.73150634765625 nanmax: 120.54721069335938 valid: True
Negative Segment - Non-Event: Case 1 lookback window: 0 min start time: 6551 end time: 6611 length: 60 sec ABP Shape: (30000,) ECG Shape: (30000,) EEG Shape: (7680,) nanmin: 58.337371826171875 nanmean: 85.06976318359375 nanmax: 133.38412475585938 valid: True
For each of the cases in my_cases_of_interest_idx overlay the results of event and segment extraction.
DISPLAY_OVERLAY_CHECK_ABP=True
DISPLAY_OVERLAY_CHECK_ABP_FIRST_ONLY=False
if DISPLAY_REALITY_CHECK_ABP:
for case_id_to_check in my_cases_of_interest_idx:
printAbpOverlay(case_id_to_check, positiveSegmentsMap,
negativeSegmentsMap, iohEventsMap, cleanEventsMap, movingAverage=True)
if DISPLAY_OVERLAY_CHECK_ABP_FIRST_ONLY:
break
Case 1 ABP Shape: (5770575, 1) nanmin: -495.6260070800781 nanmean: 78.15251159667969 nanmax: 374.3236389160156 Presence of BP > 200 valid: False
Case 4 ABP Shape: (10494700, 1) nanmin: -495.6260070800781 nanmean: 70.51863098144531 nanmax: 356.5494079589844 Presence of BP > 200 valid: False
Case 7 ABP Shape: (7884623, 1) nanmin: -495.6260070800781 nanmean: 73.2461166381836 nanmax: 342.7250061035156 Presence of BP > 200 valid: False
Case 10 ABP Shape: (10494487, 1) nanmin: -495.6260070800781 nanmean: 81.80549621582031 nanmax: 312.1138610839844 Presence of BP > 200 valid: False
Case 12 ABP Shape: (15601433, 1) nanmin: -495.6260070800781 nanmean: 71.71157836914062 nanmax: 399.0100402832031 >10% NaN valid: False
Case 13 ABP Shape: (5405175, 1) nanmin: -495.6260070800781 nanmean: 59.84370422363281 nanmax: 223.24270629882812 Presence of BP > 200 valid: False
Case 16 ABP Shape: (6432446, 1) nanmin: -495.6260070800781 nanmean: 79.41276550292969 nanmax: 406.9096984863281 Presence of BP > 200 valid: False
Case 19 ABP Shape: (13785944, 1) nanmin: -495.6260070800781 nanmean: 69.16629791259766 nanmax: 407.8971862792969 Presence of BP > 200 valid: False
Case 20 ABP Shape: (13237618, 1) nanmin: -495.6260070800781 nanmean: 75.18829345703125 nanmax: 305.2016296386719 Presence of BP > 200 valid: False
Case 22 ABP Shape: (7186505, 1) nanmin: -410.7047119140625 nanmean: 87.67796325683594 nanmax: 407.8971862792969 Presence of BP > 200 valid: False
# free memory
del tmp_abp
def get_segment_attributes_from_filename(file_path):
pieces = os.path.basename(file_path).split('_')
case = int(pieces[0])
startX = int(pieces[1])
predWindow = int(pieces[2])
label = pieces[3].replace('.h5', '')
return (case, startX, predWindow, label)
count_negative_samples = 0
count_positive_samples = 0
samples = []
from glob import glob
seg_folder = f"{VITAL_EXTRACTED_SEGMENTS}"
filenames = [y for x in os.walk(seg_folder) for y in glob(os.path.join(x[0], '*.h5'))]
for filename in filenames:
(case, start_x, pred_window, label) = get_segment_attributes_from_filename(filename)
#print((case, start_x, pred_window, label))
# only load cases for cases of interest; this folder could have segments for hundreds of cases
if case not in cases_of_interest_idx:
continue
#PREDICTION_WINDOW = 3
if pred_window == 0 or pred_window == PREDICTION_WINDOW or PREDICTION_WINDOW == 'ALL':
#print((case, start_x, pred_window, label))
if label == 'True':
count_positive_samples += 1
else:
count_negative_samples += 1
sample = (filename, label)
samples.append(sample)
print()
print(f"samples loaded: {len(samples):5} ")
print(f'count negative samples: {count_negative_samples:5}')
print(f'count positive samples: {count_positive_samples:5}')
samples loaded: 22277 count negative samples: 14125 count positive samples: 8152
# Divide by cases
sample_cases = defaultdict(lambda: [])
for fn, _ in samples:
(case, start_x, pred_window, label) = get_segment_attributes_from_filename(fn)
sample_cases[case].append((fn, label))
# understand any missing cases of interest
sample_cases_idx = pd.Index(sample_cases.keys())
missing_case_ids = cases_of_interest_idx.difference(sample_cases_idx)
print(f'cases with no samples: {missing_case_ids.shape[0]}')
print(f' {missing_case_ids}')
print()
# Split data into training, validation, and test sets
# Use 6:1:3 ratio and prevent samples from a single case from being split across different sets
# Note: number of samples at each time point is not the same, because the first event can occur before the 3/5/10/15 minute mark
# Set target sizes
train_ratio = 0.6
val_ratio = 0.1
test_ratio = 1 - train_ratio - val_ratio # ensure ratios sum to 1
# Split samples into train and other
sample_cases_train, sample_cases_other = train_test_split(list(sample_cases.keys()), test_size=(1 - train_ratio), random_state=RANDOM_SEED)
# Split other into val and test
sample_cases_val, sample_cases_test = train_test_split(sample_cases_other, test_size=(test_ratio / (1 - train_ratio)), random_state=RANDOM_SEED)
# Check how many samples are in each set
print(f'Train/Val/Test Summary by Cases')
print(f"Train cases: {len(sample_cases_train):5}, ({len(sample_cases_train) / len(sample_cases):.2%})")
print(f"Val cases: {len(sample_cases_val):5}, ({len(sample_cases_val) / len(sample_cases):.2%})")
print(f"Test cases: {len(sample_cases_test):5}, ({len(sample_cases_test) / len(sample_cases):.2%})")
print(f"Total cases: {(len(sample_cases_train) + len(sample_cases_val) + len(sample_cases_test)):5}")
cases with no samples: 251
Index([ 26, 69, 70, 126, 139, 172, 199, 218, 221, 258,
...
6109, 6131, 6180, 6192, 6205, 6261, 6275, 6293, 6330, 6339],
dtype='int64', length=251)
Train/Val/Test Summary by Cases
Train cases: 1715, (59.99%)
Val cases: 285, (9.97%)
Test cases: 859, (30.05%)
Total cases: 2859
sample_cases_train = set(sample_cases_train)
sample_cases_val = set(sample_cases_val)
sample_cases_test = set(sample_cases_test)
samples_train = []
samples_val = []
samples_test = []
for cid, segs in sample_cases.items():
if cid in sample_cases_train:
for seg in segs:
samples_train.append(seg)
if cid in sample_cases_val:
for seg in segs:
samples_val.append(seg)
if cid in sample_cases_test:
for seg in segs:
samples_test.append(seg)
# Check how many samples are in each set
print(f'Train/Val/Test Summary by Events')
print(f"Train events: {len(samples_train):5}, ({len(samples_train) / len(samples):.2%})")
print(f"Val events: {len(samples_val):5}, ({len(samples_val) / len(samples):.2%})")
print(f"Test events: {len(samples_test):5}, ({len(samples_test) / len(samples):.2%})")
print(f"Total events: {(len(samples_train) + len(samples_val) + len(samples_test)):5}")
Train/Val/Test Summary by Events Train events: 13337, (59.87%) Val events: 2187, (9.82%) Test events: 6753, (30.31%) Total events: 22277
PRINT_ALL_CASE_SPLIT_DETAILS = False
case_to_sample_distribution = defaultdict(lambda: {'train': [0, 0], 'val': [0, 0], 'test': [0, 0]})
def populate_case_to_sample_distribution(mysamples, idx):
neg = 0
pos = 0
for fn, _ in mysamples:
(case, start_x, pred_window, label) = get_segment_attributes_from_filename(fn)
slot = 0 if label == 'False' else 1
case_to_sample_distribution[case][idx][slot] += 1
if slot == 0:
neg += 1
else:
pos += 1
return (neg, pos)
train_neg, train_pos = populate_case_to_sample_distribution(samples_train, 'train')
val_neg, val_pos = populate_case_to_sample_distribution(samples_val, 'val')
test_neg, test_pos = populate_case_to_sample_distribution(samples_test, 'test')
print(f'Total Cases Present: {len(case_to_sample_distribution):5}')
print()
train_tot = train_pos + train_neg
val_tot = val_pos + val_neg
test_tot = test_pos + test_neg
print(f'Train: P: {train_pos:5} ({(train_pos/train_tot):.2}), N: {train_neg:5} ({(train_neg/train_tot):.2})')
print(f'Val: P: {val_pos:5} ({(val_pos/val_tot):.2}), N: {val_neg:5} ({(val_neg/val_tot):.2})')
print(f'Test: P: {test_pos:5} ({(test_pos/test_tot):.2}), N: {test_neg:5} ({(test_neg/test_tot):.2})')
print()
total_pos = train_pos + val_pos + test_pos
total_neg = train_neg + val_neg + test_neg
total = total_pos + total_neg
print(f'P/N Ratio: {(total_pos)}:{(total_neg)}')
print(f'P Percent: {(total_pos/total):.2}')
print(f'N Percent: {(total_neg/total):.2}')
print()
if PRINT_ALL_CASE_SPLIT_DETAILS:
for ci in sorted(case_to_sample_distribution.keys()):
print(f'{ci}: {case_to_sample_distribution[ci]}')
Total Cases Present: 2859 Train: P: 4916 (0.37), N: 8421 (0.63) Val: P: 775 (0.35), N: 1412 (0.65) Test: P: 2461 (0.36), N: 4292 (0.64) P/N Ratio: 8152:14125 P Percent: 0.37 N Percent: 0.63
def check_data_leakage(full_data, train_data, val_data, test_data):
# Convert to sets for easier operations
full_data_set = set(full_data)
train_data_set = set(train_data)
val_data_set = set(val_data)
test_data_set = set(test_data)
# Check if train, val, test are subsets of full_data
if not train_data_set.issubset(full_data_set):
return "Train data has leakage"
if not val_data_set.issubset(full_data_set):
return "Validation data has leakage"
if not test_data_set.issubset(full_data_set):
return "Test data has leakage"
# Check if train, val, test are disjoint
if train_data_set & val_data_set:
return "Train and validation data are not disjoint"
if train_data_set & test_data_set:
return "Train and test data are not disjoint"
if val_data_set & test_data_set:
return "Validation and test data are not disjoint"
return "No data leakage detected"
# Usage
print(check_data_leakage(list(sample_cases.keys()), sample_cases_train, sample_cases_val, sample_cases_test))
No data leakage detected
# Create vitalDataset class
class vitalDataset(Dataset):
def __init__(self, samples, normalize_abp=False):
self.samples = samples
self.normalize_abp = normalize_abp
def __len__(self):
return len(self.samples)
def __getitem__(self, idx):
# Get metadata for this event
segment = self.samples[idx]
file_path = segment[0]
label = (segment[1] == "True" or segment[1] == "True.vital")
(abp, ecg, eeg) = get_segment_data(file_path)
if abp is None or eeg is None or ecg is None:
return (np.zeros(30000), np.zeros(30000), np.zeros(7680), 0)
if self.normalize_abp:
abp -= 65
abp /= 65
return abp, ecg, eeg, label
NORMALIZE_ABP = False
train_dataset = vitalDataset(samples_train, NORMALIZE_ABP)
val_dataset = vitalDataset(samples_val, NORMALIZE_ABP)
test_dataset = vitalDataset(samples_test, NORMALIZE_ABP)
def generate_nan_means(mydataset):
xs = np.zeros(len(mydataset))
ys = np.zeros(len(mydataset), dtype=int)
for i, (abp, ecg, eeg, y) in enumerate(iter(mydataset)):
xs[i] = np.nanmean(abp)
ys[i] = int(y)
return pd.DataFrame({'abp_nanmean': xs, 'label': ys})
def generate_nan_means_summaries(tr, va, te, group='all'):
if group == 'all':
return pd.DataFrame({
'train': tr.describe()['abp_nanmean'],
'validation': va.describe()['abp_nanmean'],
'test': te.describe()['abp_nanmean']
})
mytr = tr.reset_index()
myva = va.reset_index()
myte = te.reset_index()
label_flag = True if group == 'positive' else False
return pd.DataFrame({
'train': mytr[mytr['label'] == label_flag].describe()['abp_nanmean'],
'validation': myva[myva['label'] == label_flag].describe()['abp_nanmean'],
'test': myte[myte['label'] == label_flag].describe()['abp_nanmean']
})
def plot_nan_means(df, plot_label):
mydf = df.reset_index()
maxCases = 'ALL' if MAX_CASES is None else MAX_CASES
plot_title = f'{plot_label} - ABP nanmean Values, {PREDICTION_WINDOW} Minutes, {maxCases} Cases'
ax = mydf[mydf['label'] == False].plot.scatter(
x='index', y='abp_nanmean', color='DarkBlue', label='Negative',
title=plot_title, figsize=(16,9))
negative_median = mydf[mydf['label'] == False]['abp_nanmean'].median()
ax.axhline(y=negative_median, color='DarkBlue', linestyle='--', label='Negative Median')
mydf[mydf['label'] == True].plot.scatter(
x='index', y='abp_nanmean', color='DarkOrange', label='Positive', ax=ax);
positive_median = mydf[mydf['label'] == True]['abp_nanmean'].median()
ax.axhline(y=positive_median, color='DarkOrange', linestyle='--', label='Positive Median')
ax.legend(loc='upper right')
def plot_nan_means_hist(df):
df.plot.hist(column=['abp_nanmean'], by='label', bins=50, figsize=(10, 8));
train_abp_nanmeans = generate_nan_means(train_dataset)
val_abp_nanmeans = generate_nan_means(val_dataset)
test_abp_nanmeans = generate_nan_means(test_dataset)
generate_nan_means_summaries(train_abp_nanmeans, val_abp_nanmeans, test_abp_nanmeans)
| train | validation | test | |
|---|---|---|---|
| count | 13337.000000 | 2187.000000 | 6753.000000 |
| mean | 84.516606 | 84.336261 | 84.230145 |
| std | 12.166553 | 12.211263 | 11.984190 |
| min | 65.093372 | 65.146761 | 65.023298 |
| 25% | 74.930521 | 74.882085 | 74.828286 |
| 50% | 82.575941 | 82.249916 | 82.292607 |
| 75% | 92.147891 | 91.899957 | 91.822852 |
| max | 147.732262 | 138.880940 | 140.253340 |
generate_nan_means_summaries(train_abp_nanmeans, val_abp_nanmeans, test_abp_nanmeans, group='positive')
| train | validation | test | |
|---|---|---|---|
| count | 4916.000000 | 775.000000 | 2461.000000 |
| mean | 77.784496 | 77.277184 | 77.526598 |
| std | 10.395182 | 10.386473 | 10.244708 |
| min | 65.093372 | 65.146761 | 65.023298 |
| 25% | 70.545436 | 70.179040 | 70.281144 |
| 50% | 74.884225 | 74.526288 | 74.864361 |
| 75% | 81.732825 | 80.940235 | 81.425981 |
| max | 146.954317 | 131.989181 | 135.669169 |
generate_nan_means_summaries(train_abp_nanmeans, val_abp_nanmeans, test_abp_nanmeans, group='negative')
| train | validation | test | |
|---|---|---|---|
| count | 8421.000000 | 1412.000000 | 4292.000000 |
| mean | 88.446668 | 88.210754 | 88.073908 |
| std | 11.378201 | 11.377544 | 11.192447 |
| min | 65.289712 | 65.588582 | 65.216607 |
| 25% | 79.911913 | 79.678075 | 79.518148 |
| 50% | 87.074796 | 86.681360 | 86.878588 |
| 75% | 95.722421 | 95.251525 | 95.323702 |
| max | 147.732262 | 138.880940 | 140.253340 |
plot_nan_means_hist(train_abp_nanmeans)
plot_nan_means_hist(val_abp_nanmeans)
plot_nan_means_hist(test_abp_nanmeans)
plot_nan_means(train_abp_nanmeans, 'Train')
plot_nan_means(val_abp_nanmeans, 'Validation')
plot_nan_means(test_abp_nanmeans, 'Test')
# Cleanup
del train_abp_nanmeans
del val_abp_nanmeans
del test_abp_nanmeans
Check if data can be easily classified using non-deep learning methods. Create a balanced sample of IOH and non-IOH events and use a simple classifier to see if the data can be easily separated. Datasets which can be easily separated by non-deep learning methods should also be easily classified by deep learning models.
MAX_CLASSIFICATION_SAMPLES = 250
MAX_SAMPLE_SIZE = 1600
classification_sample_size = MAX_SAMPLE_SIZE if len(samples) >= MAX_SAMPLE_SIZE else len(samples)
classification_samples = random.sample(samples, classification_sample_size)
positive_samples = []
negative_samples = []
for sample in classification_samples:
(sampleAbp, sampleEcg, sampleEeg) = get_segment_data(sample[0])
if sample[1] == "True":
positive_samples.append([sample[0], True, sampleAbp, sampleEcg, sampleEeg])
else:
negative_samples.append([sample[0], False, sampleAbp, sampleEcg, sampleEeg])
positive_samples = pd.DataFrame(positive_samples, columns=["file_path", "segment_label", "segment_abp", "segment_ecg", "segment_eeg"])
negative_samples = pd.DataFrame(negative_samples, columns=["file_path", "segment_label", "segment_abp", "segment_ecg", "segment_eeg"])
total_to_sample_pos = MAX_CLASSIFICATION_SAMPLES if len(positive_samples) >= MAX_CLASSIFICATION_SAMPLES else len(positive_samples)
total_to_sample_neg = MAX_CLASSIFICATION_SAMPLES if len(negative_samples) >= MAX_CLASSIFICATION_SAMPLES else len(negative_samples)
# Select up to 150 random samples where segment_label is True
positive_samples = positive_samples.sample(total_to_sample_pos, random_state=RANDOM_SEED)
# Select up to 150 random samples where segment_label is False
negative_samples = negative_samples.sample(total_to_sample_neg, random_state=RANDOM_SEED)
print(f'positive_samples: {len(positive_samples)}')
print(f'negative_samples: {len(negative_samples)}')
# Combine the positive and negative samples
samples_balanced = pd.concat([positive_samples, negative_samples])
positive_samples: 250 negative_samples: 250
Define function to build data for study. Each waveform field can be enabled or disabled:
def get_x_y(samples, use_abp, use_ecg, use_eeg):
# Create X and y, using data from `samples_balanced` and the `use_abp`, `use_ecg`, and `use_eeg` variables
X = []
y = []
for i in range(len(samples)):
row = samples.iloc[i]
sample = np.array([])
if use_abp:
if len(row['segment_abp']) != 30000:
print(len(row['segment_abp']))
sample = np.append(sample, row['segment_abp'])
if use_ecg:
if len(row['segment_ecg']) != 30000:
print(len(row['segment_ecg']))
sample = np.append(sample, row['segment_ecg'])
if use_eeg:
if len(row['segment_eeg']) != 7680:
print(len(row['segment_eeg']))
sample = np.append(sample, row['segment_eeg'])
X.append(sample)
# Convert the label from boolean to 0 or 1
y.append(int(row['segment_label']))
return X, y
Define KNN run. This is configurable to enable or disable different data channels so that we can study them individually or together:
N_NEIGHBORS = 20
def run_knn(samples, use_abp, use_ecg, use_eeg):
# Get samples
X,y = get_x_y(samples, use_abp, use_ecg, use_eeg)
# Split samples into train and val
knn_X_train, knn_X_test, knn_y_train, knn_y_test = train_test_split(X, y, test_size=0.2, random_state=RANDOM_SEED)
# Normalize the data
scaler = StandardScaler()
scaler.fit(knn_X_train)
knn_X_train = scaler.transform(knn_X_train)
knn_X_test = scaler.transform(knn_X_test)
# Initialize the KNN classifier
knn = KNeighborsClassifier(n_neighbors=N_NEIGHBORS)
# Train the KNN classifier
knn.fit(knn_X_train, knn_y_train)
# Make predictions on the test set
knn_y_pred = knn.predict(knn_X_test)
# Evaluate the KNN classifier
print(f"ABP: {use_abp}, ECG: {use_ecg}, EEG: {use_eeg}")
print(f"Confusion matrix:\n{confusion_matrix(knn_y_test, knn_y_pred)}")
print(f"Classification report:\n{classification_report(knn_y_test, knn_y_pred)}")
Study each waveform independently, then ABP+EEG (which had best results in paper), and ABP+ECG+EEG:
run_knn(samples_balanced, use_abp=True, use_ecg=False, use_eeg=False)
run_knn(samples_balanced, use_abp=False, use_ecg=True, use_eeg=False)
run_knn(samples_balanced, use_abp=False, use_ecg=False, use_eeg=True)
run_knn(samples_balanced, use_abp=True, use_ecg=False, use_eeg=True)
run_knn(samples_balanced, use_abp=True, use_ecg=True, use_eeg=True)
ABP: True, ECG: False, EEG: False
Confusion matrix:
[[38 16]
[ 9 37]]
Classification report:
precision recall f1-score support
0 0.81 0.70 0.75 54
1 0.70 0.80 0.75 46
accuracy 0.75 100
macro avg 0.75 0.75 0.75 100
weighted avg 0.76 0.75 0.75 100
ABP: False, ECG: True, EEG: False
Confusion matrix:
[[53 1]
[46 0]]
Classification report:
precision recall f1-score support
0 0.54 0.98 0.69 54
1 0.00 0.00 0.00 46
accuracy 0.53 100
macro avg 0.27 0.49 0.35 100
weighted avg 0.29 0.53 0.37 100
ABP: False, ECG: False, EEG: True
Confusion matrix:
[[ 0 54]
[ 2 44]]
Classification report:
precision recall f1-score support
0 0.00 0.00 0.00 54
1 0.45 0.96 0.61 46
accuracy 0.44 100
macro avg 0.22 0.48 0.31 100
weighted avg 0.21 0.44 0.28 100
ABP: True, ECG: False, EEG: True
Confusion matrix:
[[39 15]
[11 35]]
Classification report:
precision recall f1-score support
0 0.78 0.72 0.75 54
1 0.70 0.76 0.73 46
accuracy 0.74 100
macro avg 0.74 0.74 0.74 100
weighted avg 0.74 0.74 0.74 100
ABP: True, ECG: True, EEG: True
Confusion matrix:
[[39 15]
[11 35]]
Classification report:
precision recall f1-score support
0 0.78 0.72 0.75 54
1 0.70 0.76 0.73 46
accuracy 0.74 100
macro avg 0.74 0.74 0.74 100
weighted avg 0.74 0.74 0.74 100
Based on the data above, the ABP data alone is strongly predictive based on the macro average F1-score of 0.90. The ECG and EEG data are weakly predictive with F1 scores of 0.33 and 0.64, respectively. The ABP+EEG data is also strongly predictive with an F1 score of 0.88, and ABP+ECG+EEG data somewhat predictive with an F1 score of 0.79.
Models based on ABP data alone, or ABP+EEG data are expected to train easily with good performance. The other signals appear to mostly add noise and are not strongly predictive. This agrees with the results from the paper.
Define t-SNE run. This is configurable to enable or disable different data channels so that we can study them individually or together:
def run_tsne(samples, use_abp, use_ecg, use_eeg):
# Get samples
X,y = get_x_y(samples, use_abp, use_ecg, use_eeg)
# Convert X and y to numpy arrays
X = np.array(X)
y = np.array(y)
# Run t-SNE on the samples
tsne = TSNE(n_components=len(np.unique(y)), random_state=RANDOM_SEED)
X_tsne = tsne.fit_transform(X)
# Create a scatter plot of the t-SNE representation
plt.figure(figsize=(16, 9))
plt.title(f"use_abp={use_abp}, use_ecg={use_ecg}, use_eeg={use_eeg}")
for i, label in enumerate(set(y)):
plt.scatter(X_tsne[y == label, 0], X_tsne[y == label, 1], label=label)
plt.legend()
plt.show()
Study each waveform independently, then ABP+EEG (which had best results in paper), and ABP+ECG+EEG:
run_tsne(samples_balanced, use_abp=True, use_ecg=False, use_eeg=False)
run_tsne(samples_balanced, use_abp=False, use_ecg=True, use_eeg=False)
run_tsne(samples_balanced, use_abp=False, use_ecg=False, use_eeg=True)
run_tsne(samples_balanced, use_abp=True, use_ecg=False, use_eeg=True)
run_tsne(samples_balanced, use_abp=True, use_ecg=True, use_eeg=True)
Based on the plots above, it appears that ABP alone, ABP+EEG and ABP+ECG+EEG are somewhat separable, though with outliers, and should be trainable by our model. The ECG and EEG data are not easily separable from the other data. This agrees with the results from the paper.
# cleanup
del samples_balanced
The model implementation is based on the CNN architecture described in Jo Y-Y et al. (2022). It is designed to handle 1, 2, or 3 signal categories simultaneously, allowing for flexible model configurations based on different combinations of physiological signals:
The architecture, as depicted in Figure 2 from the original paper, utilizes a ResNet-based approach tailored for time-series data from different physiological signals. The model architecture is adapted to handle varying input signal frequencies, with specific hyperparameters for each signal type, particularly EEG, due to its distinct characteristics compared to ABP and ECG. A diagram of the model architecture is shown below:
Each input signal is processed through a sequence of 12 7-layer residual blocks, followed by a flattening process and a linear transformation to produce a 32-dimensional feature vector per signal type. These vectors are then concatenated (if multiple signals are used) and passed through two additional linear layers to produce a single output vector, representing the IOH index. A threshold is determined experimentally in order to minimize the differene between the sensitivity and specificity and is applied to this index to perform binary classification for predicting IOH events.
The hyperparameters for the residual blocks are specified in Supplemental Table 1 from the original paper and vary for different signal type.
A forward pass through the model passes through 85 layers before concatenation, followed by two more linear layers and finally a sigmoid activation layer to produce the prediction measure.
Each residual block consists of the following seven layers:
Skip connections are included to aid in gradient flow during training, with optional 1D convolution in the skip connection to align dimensions.
The hyperparameters are detailed in Supplemental Table 1 of the original paper. A screenshot of these hyperparameters is provided for reference below:

Note: Please be aware of a transcription error in the original paper's Supplemental Table 1 for the ECG+ABP configuration in Residual Blocks 11 and 12, where the output size should be 469 6 instead of the reported 496 6.
Our model uses binary cross entropy as the loss function and Adam as the optimizer, consistent with the original study. The learning rate is set at 0.0001, and training is configured to run for up to 100 epochs, with early stopping implemented if no improvement in loss is observed over five consecutive epochs.
# First define the residual block which is reused 12x for each data track for each sample.
# Second define the primary model.
class ResidualBlock(nn.Module):
def __init__(self, in_features: int, out_features: int, in_channels: int, out_channels: int, kernel_size: int, stride: int = 1, size_down: bool = False, ignoreSkipConnection: bool = False) -> None:
super(ResidualBlock, self).__init__()
self.ignoreSkipConnection = ignoreSkipConnection
# calculate the appropriate padding required to ensure expected sequence lengths out of each residual block
padding = int((((stride-1)*in_features)-stride+kernel_size)/2)
self.size_down = size_down
self.bn1 = nn.BatchNorm1d(in_channels)
self.relu = nn.ReLU()
self.dropout = nn.Dropout(0.5)
self.conv1 = nn.Conv1d(in_channels, out_channels, kernel_size=kernel_size, stride=1, padding=padding, bias=False)
self.bn2 = nn.BatchNorm1d(out_channels)
self.conv2 = nn.Conv1d(out_channels, out_channels, kernel_size=kernel_size, stride=1, padding=padding, bias=False)
self.residualConv = nn.Conv1d(in_channels, out_channels, kernel_size=kernel_size, stride=1, padding=padding, bias=False)
# unclear where in sequence this should take place. Size down expressed in Supplemental table S1
if self.size_down:
pool_padding = (1 if (in_features % 2 > 0) else 0)
self.downsample = nn.MaxPool1d(kernel_size=2, stride=2, padding = pool_padding)
def forward(self, x: torch.Tensor) -> torch.Tensor:
identity = x
out = self.bn1(x)
out = self.relu(out)
out = self.dropout(out)
out = self.conv1(out)
if self.size_down:
out = self.downsample(out)
out = self.bn2(out)
out = self.relu(out)
out = self.conv2(out)
if not self.ignoreSkipConnection:
if out.shape != identity.shape:
# run the residual through a convolution when necessary
identity = self.residualConv(identity)
outlen = np.prod(out.shape)
idlen = np.prod(identity.shape)
# downsample when required
if idlen > outlen:
identity = self.downsample(identity)
# match dimensions
identity = identity.reshape(out.shape)
# add the residual
out += identity
return out
class HypotensionCNN(nn.Module):
def __init__(self, useAbp: bool = True, useEeg: bool = False, useEcg: bool = False, device: str = "cpu", nResiduals: int = 12, ignoreSkipConnection: bool = False, useSigmoid: bool = True) -> None:
assert useAbp or useEeg or useEcg, "At least one data track must be used"
assert nResiduals > 0 and nResiduals <= 12, "Number of residual blocks must be between 1 and 12"
super(HypotensionCNN, self).__init__()
self.device = device
self.useAbp = useAbp
self.useEeg = useEeg
self.useEcg = useEcg
self.nResiduals = nResiduals
self.useSigmoid = useSigmoid
# Size of the concatenated output from the residual blocks
concatSize = 0
if useAbp:
self.abpBlocks = []
self.abpMultipliers = [1, 2, 2, 2, 2, 2, 4, 4, 4, 4, 4, 6, 6]
self.abpSizes = [30000, 15000, 15000, 7500, 7500, 3750, 3750, 1875, 1875, 938, 938, 469, 469]
for i in range(self.nResiduals):
downsample = i % 2 == 0
self.abpBlocks.append(ResidualBlock(self.abpSizes[i], self.abpSizes[i+1], self.abpMultipliers[i], self.abpMultipliers[i+1], 15 if i < 6 else 7, 1, downsample, ignoreSkipConnection))
self.abpResiduals = nn.Sequential(*self.abpBlocks)
self.abpFc = nn.Linear(self.abpMultipliers[self.nResiduals] * self.abpSizes[self.nResiduals], 32)
concatSize += 32
if useEcg:
self.ecgBlocks = []
self.ecgMultipliers = [1, 2, 2, 2, 2, 2, 4, 4, 4, 4, 4, 6, 6]
self.ecgSizes = [30000, 15000, 15000, 7500, 7500, 3750, 3750, 1875, 1875, 938, 938, 469, 469]
for i in range(self.nResiduals):
downsample = i % 2 == 0
self.ecgBlocks.append(ResidualBlock(self.ecgSizes[i], self.ecgSizes[i+1], self.ecgMultipliers[i], self.ecgMultipliers[i+1], 15 if i < 6 else 7, 1, downsample, ignoreSkipConnection))
self.ecgResiduals = nn.Sequential(*self.ecgBlocks)
self.ecgFc = nn.Linear(self.ecgMultipliers[self.nResiduals] * self.ecgSizes[self.nResiduals], 32)
concatSize += 32
if useEeg:
self.eegBlocks = []
self.eegMultipliers = [1, 2, 2, 2, 2, 2, 4, 4, 4, 4, 4, 6, 6]
self.eegSizes = [7680, 3840, 3840, 1920, 1920, 960, 960, 480, 480, 240, 240, 120, 120]
for i in range(self.nResiduals):
downsample = i % 2 == 0
self.eegBlocks.append(ResidualBlock(self.eegSizes[i], self.eegSizes[i+1], self.eegMultipliers[i], self.eegMultipliers[i+1], 7 if i < 6 else 3, 1, downsample, ignoreSkipConnection))
self.eegResiduals = nn.Sequential(*self.eegBlocks)
self.eegFc = nn.Linear(self.eegMultipliers[self.nResiduals] * self.eegSizes[self.nResiduals], 32)
concatSize += 32
self.fullLinear1 = nn.Linear(concatSize, 16)
self.fullLinear2 = nn.Linear(16, 1)
self.sigmoid = nn.Sigmoid()
def forward(self, abp: torch.Tensor, eeg: torch.Tensor, ecg: torch.Tensor) -> torch.Tensor:
batchSize = len(abp)
# conditionally operate ABP, EEG, and ECG networks
tensors = []
if self.useAbp:
self.abpResiduals.to(self.device)
abp = self.abpResiduals(abp)
totalLen = np.prod(abp.shape)
abp = torch.reshape(abp, (batchSize, int(totalLen / batchSize)))
abp = self.abpFc(abp)
tensors.append(abp)
if self.useEeg:
self.eegResiduals.to(self.device)
eeg = self.eegResiduals(eeg)
totalLen = np.prod(eeg.shape)
eeg = torch.reshape(eeg, (batchSize, int(totalLen / batchSize)))
eeg = self.eegFc(eeg)
tensors.append(eeg)
if self.useEcg:
self.ecgResiduals.to(self.device)
ecg = self.ecgResiduals(ecg)
totalLen = np.prod(ecg.shape)
ecg = torch.reshape(ecg, (batchSize, int(totalLen / batchSize)))
ecg = self.ecgFc(ecg)
tensors.append(ecg)
# concatenate the tensors along dimension 1 if there's more than one, otherwise use the single tensor
merged = torch.cat(tensors, dim=1) if len(tensors) > 1 else tensors[0]
totalLen = np.prod(merged.shape)
merged = torch.reshape(merged, (batchSize, int(totalLen / batchSize)))
out = self.fullLinear1(merged)
out = self.fullLinear2(out)
if self.useSigmoid:
out = self.sigmoid(out)
# We should not be seeing NaNs! If we are, there is a problem upstream.
#out = torch.nan_to_num(out)
return out
As discussed earlier, our model uses binary cross entropy as the loss function and Adam as the optimizer, consistent with the original study. The learning rate is set at 0.0001, and training is configured to run for up to 100 epochs, with early stopping implemented if no improvement in loss is observed over five consecutive epochs.
def train_model_one_iter(model, device, loss_func, optimizer, train_loader):
model.train()
train_losses = []
for abp, ecg, eeg, label in tqdm(train_loader):
batch = len(abp)
abp = abp.reshape(batch, 1, -1).type(torch.FloatTensor).to(device)
ecg = ecg.reshape(batch, 1, -1).type(torch.FloatTensor).to(device)
eeg = eeg.reshape(batch, 1, -1).type(torch.FloatTensor).to(device)
label = label.type(torch.float).reshape(batch, 1).to(device)
optimizer.zero_grad()
mdl = model(abp, eeg, ecg)
loss = loss_func(torch.nan_to_num(mdl), label)
loss.backward()
optimizer.step()
train_losses.append(loss.cpu().data.numpy())
return np.mean(train_losses)
def evaluate_model(model, loss_func, val_loader):
model.eval()
val_losses = []
for abp, ecg, eeg, label in tqdm(val_loader):
batch = len(abp)
abp = abp.reshape(batch, 1, -1).type(torch.FloatTensor).to(device)
ecg = ecg.reshape(batch, 1, -1).type(torch.FloatTensor).to(device)
eeg = eeg.reshape(batch, 1, -1).type(torch.FloatTensor).to(device)
label = label.type(torch.float).reshape(batch, 1).to(device)
mdl = model(abp, eeg, ecg)
loss = loss_func(torch.nan_to_num(mdl), label)
val_losses.append(loss.cpu().data.numpy())
return np.mean(val_losses)
def plot_losses(train_losses, val_losses, best_epoch, experimentName):
print()
print(f'Plot Validation and Loss Values from Training')
print(f' Epoch with best Validation Loss: {best_epoch:3}, {val_losses[best_epoch]:.4}')
# Create x-axis values for epochs
epochs = range(0, len(train_losses))
plt.figure(figsize=(16, 9))
# Plot the training and validation losses
plt.plot(epochs, train_losses, 'b', label='Training Loss')
plt.plot(epochs, val_losses, 'r', label='Validation Loss')
# Add a vertical bar at the best_epoch
plt.axvline(x=best_epoch, color='g', linestyle='--', label='Best Epoch')
# Shade everything to the right of the best_epoch a light red
plt.axvspan(best_epoch, max(epochs), facecolor='r', alpha=0.1)
# Add labels and title
plt.xlabel('Epochs')
plt.ylabel('Loss')
plt.title(experimentName)
# Add legend
plt.legend(loc='upper right')
# Show the plot
plt.show()
def eval_model(model, device, dataloader, loss_func, print_detailed: bool = False):
model.eval()
model = model.to(device)
total_loss = 0
all_predictions = []
all_labels = []
with torch.no_grad():
for abp, ecg, eeg, label in tqdm(dataloader):
batch = len(abp)
abp = torch.nan_to_num(abp.reshape(batch, 1, -1)).type(torch.FloatTensor).to(device)
ecg = torch.nan_to_num(ecg.reshape(batch, 1, -1)).type(torch.FloatTensor).to(device)
eeg = torch.nan_to_num(eeg.reshape(batch, 1, -1)).type(torch.FloatTensor).to(device)
label = label.type(torch.float).reshape(batch, 1).to(device)
pred = model(abp, eeg, ecg)
loss = loss_func(pred, label)
total_loss += loss.item()
all_predictions.append(pred.detach().cpu().numpy())
all_labels.append(label.detach().cpu().numpy())
# Flatten the lists
all_predictions = np.concatenate(all_predictions).flatten()
all_labels = np.concatenate(all_labels).flatten()
# Calculate AUROC and AUPRC
# y_true, y_pred
auroc = roc_auc_score(all_labels, all_predictions)
precision, recall, _ = precision_recall_curve(all_labels, all_predictions)
auprc = auc(recall, precision)
# Determine the optimal threshold, which is argmin(abs(sensitivity - specificity)) per the paper
thresholds = np.linspace(0, 1, 101) # 0 to 1 in 0.01 steps
min_diff = float('inf')
optimal_sensitivity = None
optimal_specificity = None
optimal_threshold = None
for threshold in thresholds:
all_predictions_binary = (all_predictions > threshold).astype(int)
tn, fp, fn, tp = confusion_matrix(all_labels, all_predictions_binary).ravel()
sensitivity = tp / (tp + fn)
specificity = tn / (tn + fp)
diff = abs(sensitivity - specificity)
if diff < min_diff:
min_diff = diff
optimal_threshold = threshold
optimal_sensitivity = sensitivity
optimal_specificity = specificity
avg_loss = total_loss / len(dataloader)
if print_detailed:
print(f"Predictions: {all_predictions}")
print(f"Labels: {all_labels}")
print(f"Loss: {avg_loss}")
print(f"AUROC: {auroc}")
print(f"AUPRC: {auprc}")
print(f"Sensitivity: {optimal_sensitivity}")
print(f"Specificity: {optimal_specificity}")
print(f"Threshold: {optimal_threshold}")
return all_predictions, all_labels, avg_loss, auroc, auprc, optimal_sensitivity, optimal_specificity, optimal_threshold
def print_all_evals(model, models, device, val_loader, test_loader, loss_func, print_detailed: bool = False):
print()
print(f'Generate AUROC/AUPRC for Each Intermediate Model')
print()
val_aurocs = []
val_auprcs = []
test_aurocs = []
test_auprcs = []
for mod in models:
model.load_state_dict(torch.load(mod))
model.train(False)
print(f'Intermediate Model:')
print(f' {mod}')
# validation loop
print("AUROC/AUPRC on Validation Data")
_, _, _, valid_auroc, valid_auprc, _, _, _ = \
eval_model(model, device, val_loader, loss_func, print_detailed)
val_aurocs.append(valid_auroc)
val_auprcs.append(valid_auprc)
print()
# test loop
print("AUROC/AUPRC on Test Data")
_, _, _, test_auroc, test_auprc, _, _, _ = \
eval_model(model, device, test_loader, loss_func, print_detailed)
test_aurocs.append(test_auroc)
test_auprcs.append(test_auprc)
print()
return val_aurocs, val_auprcs, test_aurocs, test_auprcs
def plot_auroc_auprc(val_losses, val_aurocs, val_auprcs, test_aurocs, test_auprcs, all_models, best_epoch):
print()
print(f'Plot AUROC/AUPRC for Each Intermediate Model')
# Create x-axis values for epochs
epochs = range(0, len(val_aurocs))
# Find model with highest AUROC
np_test_aurocs = np.array(test_aurocs)
test_auroc_idx = np.argmax(np_test_aurocs)
print(f' Epoch with best Validation Loss: {best_epoch:3}, {val_losses[best_epoch]:.4}')
print(f' Epoch with best model Test AUROC: {test_auroc_idx:3}, {np.max(np_test_aurocs):.4}')
#print(f'Best Model on Validation Loss:')
#print(f' {all_models[test_auroc_idx]}')
#print(f'Best Model on Test AUROC:')
#print(f' {all_models[best_epoch]}')
plt.figure(figsize=(16, 9))
# Plot the training and validation losses
plt.plot(epochs, val_aurocs, 'C0', label='AUROC - Validation')
plt.plot(epochs, test_aurocs, 'C1', label='AUROC - Test')
plt.plot(epochs, val_auprcs, 'C2', label='AUPRC - Validation')
plt.plot(epochs, test_auprcs, 'C3', label='AUPRC - Test')
# Add a vertical bar at the best_epoch
plt.axvline(x=best_epoch, color='g', linestyle='--', label='Best Epoch - Validation Loss')
plt.axvline(x=test_auroc_idx, color='maroon', linestyle='--', label='Best Epoch - Test AUROC')
# Shade everything to the right of the best_model a light red
plt.axvspan(test_auroc_idx, max(epochs), facecolor='r', alpha=0.1)
# Add labels and title
plt.xlabel('Epochs')
plt.ylabel('AUROC / AUPRC')
plt.title('Validation and Test AUROC by Model Iteration Across Training')
# Add legend
plt.legend(loc='right')
# Show the plot
plt.show()
return np_test_aurocs, test_auroc_idx
def run_experiment(
experimentNamePrefix: str = None,
useAbp: bool = True,
useEeg: bool = False,
useEcg: bool = False,
nResiduals: int = 12,
skip_connection: bool = False,
batch_size: int = 64,
learning_rate: float = 1e-4,
weight_decay: float = 0.0,
balance_labels: bool = False,
pos_weight: float = None,
max_epochs: int = 100,
patience: int = 25,
device: str = "cpu"
):
experimentName = ""
experimentOptions = [experimentNamePrefix, 'ABP', 'EEG', 'ECG', 'SKIPCONNECTION']
experimentValues = [experimentNamePrefix is not None, useAbp, useEeg, useEcg, skip_connection]
experimentFlags = [name for name, value in zip(experimentOptions, experimentValues) if value]
if experimentFlags:
experimentName = "_".join(experimentFlags)
experimentName = f"{experimentName}_{nResiduals}_RESIDUAL_BLOCKS_{batch_size}_BATCH_SIZE_{learning_rate}_LEARNING_RATE"
if weight_decay is not None and weight_decay != 0.0:
experimentName = f"{experimentName}_{weight_decay}_WEIGHT_DECAY"
predictionWindow = 'ALL' if PREDICTION_WINDOW == 'ALL' else f'{PREDICTION_WINDOW:03}'
experimentName = f"{experimentName}_{predictionWindow}_MINS"
maxCases = '_ALL' if MAX_CASES is None else f'{MAX_CASES:04}'
experimentName = f"{experimentName}_{maxCases}_MAX_CASES"
# default label split based on empirical data
my_pos_weight = 4.0
if balance_labels and pos_weight is not None:
my_pos_weight = pos_weight
print(f"Experiment Setup")
print(f' name: {experimentName}')
print(f' prediction_window: {predictionWindow}')
print(f' max_cases: {maxCases}')
print(f' use_abp: {useAbp}')
print(f' use_eeg: {useEeg}')
print(f' use_ecg: {useEcg}')
print(f' n_residuals: {nResiduals}')
print(f' skip_connection: {skip_connection}')
print(f' batch_size: {batch_size}')
print(f' learning_rate: {learning_rate}')
print(f' weight_decay: {weight_decay}')
print(f' balance_labels: {balance_labels}')
if balance_labels:
print(f' pos_weight: {my_pos_weight}')
print(f' max_epochs: {max_epochs}')
print(f' patience: {patience}')
print(f' device: {device}')
print()
train_loader = torch.utils.data.DataLoader(train_dataset, batch_size=batch_size, shuffle=True)
val_loader = torch.utils.data.DataLoader(val_dataset, batch_size=batch_size, shuffle=True)
test_loader = torch.utils.data.DataLoader(test_dataset, batch_size=batch_size, shuffle=False)
# Disable final sigmoid activation for BCEWithLogitsLoss
model = HypotensionCNN(useAbp, useEeg, useEcg, device, nResiduals, skip_connection, useSigmoid=(not balance_labels))
model = model.to(device)
if balance_labels:
# Only the weight for the positive class
loss_func = nn.BCEWithLogitsLoss(pos_weight=torch.tensor([my_pos_weight]).to(device))
else:
loss_func = nn.BCELoss()
optimizer = torch.optim.Adam(model.parameters(), lr=learning_rate, weight_decay=weight_decay)
print(f'Model Architecture')
print(model)
print()
print(f'Training Loop')
# Training loop
best_epoch = 0
train_losses = []
val_losses = []
best_loss = float('inf')
no_improve_epochs = 0
model_path = os.path.join(VITAL_MODELS, f"{experimentName}.model")
all_models = []
for i in range(max_epochs):
# Train the model and get the training loss
train_loss = train_model_one_iter(model, device, loss_func, optimizer, train_loader)
train_losses.append(train_loss)
# Calculate validate loss
val_loss = evaluate_model(model, loss_func, val_loader)
val_losses.append(val_loss)
print(f"[{datetime.now()}] Completed epoch {i} with training loss {train_loss:.8f}, validation loss {val_loss:.8f}")
# Save all intermediary models.
tmp_model_path = os.path.join(VITAL_MODELS, f"{experimentName}_{i:04d}.model")
torch.save(model.state_dict(), tmp_model_path)
all_models.append(tmp_model_path)
# Check if validation loss has improved
if val_loss < best_loss:
best_epoch = i
best_loss = val_loss
no_improve_epochs = 0
torch.save(model.state_dict(), model_path)
print(f"Validation loss improved to {val_loss:.8f}. Model saved.")
else:
no_improve_epochs += 1
print(f"No improvement in validation loss. {no_improve_epochs} epochs without improvement.")
# exit early if no improvement in loss over last 'patience' epochs
if no_improve_epochs >= patience:
print("Early stopping due to no improvement in validation loss.")
break
# Load best model from disk
#print()
#if os.path.exists(model_path):
# model.load_state_dict(torch.load(model_path))
# print(f"Loaded best model from disk from epoch {best_epoch}.")
#else:
# print("No saved model found for f{experimentName}.")
model.train(False)
# Plot the training and validation losses across all training epochs.
plot_losses(train_losses, val_losses, best_epoch, experimentName)
# Generate AUROC/AUPRC for each intermediate model generated across training epochs.
val_aurocs, val_auprcs, test_aurocs, test_auprcs = \
print_all_evals(model, all_models, device, val_loader, test_loader, loss_func, print_detailed=False)
# Find model with highest AUROC. Plot AUROC/AUPRC across all epochs.
np_test_aurocs, test_auroc_idx = plot_auroc_auprc(val_losses, val_aurocs, val_auprcs, \
test_aurocs, test_auprcs, all_models, best_epoch)
## AUROC / AUPRC - Model with Best Validation Loss
best_model_val_loss = all_models[best_epoch]
print(f'AUROC/AUPRC Plots - Best Model Based on Validation Loss')
print(f' Epoch with best Validation Loss: {best_epoch:3}, {val_losses[best_epoch]:.4}')
print(f' Best Model Based on Validation Loss:')
print(f' {best_model_val_loss}')
print()
print(f'Generate Stats Based on Test Data')
model.load_state_dict(torch.load(best_model_val_loss))
model.train(False)
best_model_val_test_predictions, best_model_val_test_labels, test_loss, \
best_model_val_test_auroc, best_model_val_test_auprc, test_sensitivity, test_specificity, \
best_model_val_test_threshold = eval_model(model, device, test_loader, loss_func, print_detailed=False)
# y_test, y_pred
display = RocCurveDisplay.from_predictions(
best_model_val_test_labels,
best_model_val_test_predictions,
plot_chance_level=True
)
plt.show()
print(f'best_model_val_test_auroc: {best_model_val_test_auroc}')
best_model_val_test_predictions_binary = \
(best_model_val_test_predictions > best_model_val_test_threshold).astype(int)
# y_test, y_pred
display = PrecisionRecallDisplay.from_predictions(
best_model_val_test_labels,
best_model_val_test_predictions_binary,
plot_chance_level=True
)
plt.show()
print(f'best_model_val_test_auprc: {best_model_val_test_auprc}')
print()
## AUROC / AUPRC - Model with Best AUROC
# Find model with highest AUROC
best_model_auroc = all_models[test_auroc_idx]
print(f'AUROC/AUPRC Plots - Best Model Based on Model AUROC')
print(f' Epoch with best model Test AUROC: {test_auroc_idx:3}, {np.max(np_test_aurocs):.4}')
print(f' Best Model Based on Model AUROC:')
print(f' {best_model_auroc}')
print()
print(f'Generate Stats Based on Test Data')
model.load_state_dict(torch.load(best_model_auroc))
model.train(False)
best_model_auroc_test_predictions, best_model_auroc_test_labels, test_loss, \
best_model_auroc_test_auroc, best_model_auroc_test_auprc, test_sensitivity, test_specificity, \
best_model_auroc_test_threshold = eval_model(model, device, test_loader, loss_func, print_detailed=False)
# y_test, y_pred
display = RocCurveDisplay.from_predictions(
best_model_auroc_test_labels,
best_model_auroc_test_predictions,
plot_chance_level=True
)
plt.show()
print(f'best_model_auroc_test_auroc: {best_model_auroc_test_auroc}')
best_model_auroc_test_predictions_binary = \
(best_model_auroc_test_predictions > best_model_auroc_test_threshold).astype(int)
# y_test, y_pred
display = PrecisionRecallDisplay.from_predictions(
best_model_auroc_test_labels,
best_model_auroc_test_predictions_binary,
plot_chance_level=True
)
plt.show()
print(f"best_model_auroc_test_auprc: {best_model_auroc_test_auprc}")
print('Time to experiment!')
Time to experiment!
run_experiment(
experimentNamePrefix=None,
useAbp=True,
useEeg=False,
useEcg=False,
nResiduals=12,
skip_connection=False,
batch_size=64,
learning_rate=1e-4,
weight_decay=0.0,
balance_labels=False,
#pos_weight=2.0,
pos_weight=None,
max_epochs=100,
patience=15,
device=device
)
Experiment Setup
name: ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES
prediction_window: 003
max_cases: _ALL
use_abp: True
use_eeg: False
use_ecg: False
n_residuals: 12
skip_connection: False
batch_size: 64
learning_rate: 0.0001
weight_decay: 0.0
balance_labels: False
max_epochs: 100
patience: 15
device: mps
Model Architecture
HypotensionCNN(
(abpResiduals): Sequential(
(0): ResidualBlock(
(bn1): BatchNorm1d(1, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU()
(dropout): Dropout(p=0.5, inplace=False)
(conv1): Conv1d(1, 2, kernel_size=(15,), stride=(1,), padding=(7,), bias=False)
(bn2): BatchNorm1d(2, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(conv2): Conv1d(2, 2, kernel_size=(15,), stride=(1,), padding=(7,), bias=False)
(residualConv): Conv1d(1, 2, kernel_size=(15,), stride=(1,), padding=(7,), bias=False)
(downsample): MaxPool1d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
)
(1): ResidualBlock(
(bn1): BatchNorm1d(2, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU()
(dropout): Dropout(p=0.5, inplace=False)
(conv1): Conv1d(2, 2, kernel_size=(15,), stride=(1,), padding=(7,), bias=False)
(bn2): BatchNorm1d(2, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(conv2): Conv1d(2, 2, kernel_size=(15,), stride=(1,), padding=(7,), bias=False)
(residualConv): Conv1d(2, 2, kernel_size=(15,), stride=(1,), padding=(7,), bias=False)
)
(2): ResidualBlock(
(bn1): BatchNorm1d(2, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU()
(dropout): Dropout(p=0.5, inplace=False)
(conv1): Conv1d(2, 2, kernel_size=(15,), stride=(1,), padding=(7,), bias=False)
(bn2): BatchNorm1d(2, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(conv2): Conv1d(2, 2, kernel_size=(15,), stride=(1,), padding=(7,), bias=False)
(residualConv): Conv1d(2, 2, kernel_size=(15,), stride=(1,), padding=(7,), bias=False)
(downsample): MaxPool1d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
)
(3): ResidualBlock(
(bn1): BatchNorm1d(2, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU()
(dropout): Dropout(p=0.5, inplace=False)
(conv1): Conv1d(2, 2, kernel_size=(15,), stride=(1,), padding=(7,), bias=False)
(bn2): BatchNorm1d(2, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(conv2): Conv1d(2, 2, kernel_size=(15,), stride=(1,), padding=(7,), bias=False)
(residualConv): Conv1d(2, 2, kernel_size=(15,), stride=(1,), padding=(7,), bias=False)
)
(4): ResidualBlock(
(bn1): BatchNorm1d(2, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU()
(dropout): Dropout(p=0.5, inplace=False)
(conv1): Conv1d(2, 2, kernel_size=(15,), stride=(1,), padding=(7,), bias=False)
(bn2): BatchNorm1d(2, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(conv2): Conv1d(2, 2, kernel_size=(15,), stride=(1,), padding=(7,), bias=False)
(residualConv): Conv1d(2, 2, kernel_size=(15,), stride=(1,), padding=(7,), bias=False)
(downsample): MaxPool1d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
)
(5): ResidualBlock(
(bn1): BatchNorm1d(2, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU()
(dropout): Dropout(p=0.5, inplace=False)
(conv1): Conv1d(2, 4, kernel_size=(15,), stride=(1,), padding=(7,), bias=False)
(bn2): BatchNorm1d(4, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(conv2): Conv1d(4, 4, kernel_size=(15,), stride=(1,), padding=(7,), bias=False)
(residualConv): Conv1d(2, 4, kernel_size=(15,), stride=(1,), padding=(7,), bias=False)
)
(6): ResidualBlock(
(bn1): BatchNorm1d(4, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU()
(dropout): Dropout(p=0.5, inplace=False)
(conv1): Conv1d(4, 4, kernel_size=(7,), stride=(1,), padding=(3,), bias=False)
(bn2): BatchNorm1d(4, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(conv2): Conv1d(4, 4, kernel_size=(7,), stride=(1,), padding=(3,), bias=False)
(residualConv): Conv1d(4, 4, kernel_size=(7,), stride=(1,), padding=(3,), bias=False)
(downsample): MaxPool1d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
)
(7): ResidualBlock(
(bn1): BatchNorm1d(4, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU()
(dropout): Dropout(p=0.5, inplace=False)
(conv1): Conv1d(4, 4, kernel_size=(7,), stride=(1,), padding=(3,), bias=False)
(bn2): BatchNorm1d(4, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(conv2): Conv1d(4, 4, kernel_size=(7,), stride=(1,), padding=(3,), bias=False)
(residualConv): Conv1d(4, 4, kernel_size=(7,), stride=(1,), padding=(3,), bias=False)
)
(8): ResidualBlock(
(bn1): BatchNorm1d(4, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU()
(dropout): Dropout(p=0.5, inplace=False)
(conv1): Conv1d(4, 4, kernel_size=(7,), stride=(1,), padding=(3,), bias=False)
(bn2): BatchNorm1d(4, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(conv2): Conv1d(4, 4, kernel_size=(7,), stride=(1,), padding=(3,), bias=False)
(residualConv): Conv1d(4, 4, kernel_size=(7,), stride=(1,), padding=(3,), bias=False)
(downsample): MaxPool1d(kernel_size=2, stride=2, padding=1, dilation=1, ceil_mode=False)
)
(9): ResidualBlock(
(bn1): BatchNorm1d(4, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU()
(dropout): Dropout(p=0.5, inplace=False)
(conv1): Conv1d(4, 4, kernel_size=(7,), stride=(1,), padding=(3,), bias=False)
(bn2): BatchNorm1d(4, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(conv2): Conv1d(4, 4, kernel_size=(7,), stride=(1,), padding=(3,), bias=False)
(residualConv): Conv1d(4, 4, kernel_size=(7,), stride=(1,), padding=(3,), bias=False)
)
(10): ResidualBlock(
(bn1): BatchNorm1d(4, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU()
(dropout): Dropout(p=0.5, inplace=False)
(conv1): Conv1d(4, 6, kernel_size=(7,), stride=(1,), padding=(3,), bias=False)
(bn2): BatchNorm1d(6, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(conv2): Conv1d(6, 6, kernel_size=(7,), stride=(1,), padding=(3,), bias=False)
(residualConv): Conv1d(4, 6, kernel_size=(7,), stride=(1,), padding=(3,), bias=False)
(downsample): MaxPool1d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
)
(11): ResidualBlock(
(bn1): BatchNorm1d(6, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(relu): ReLU()
(dropout): Dropout(p=0.5, inplace=False)
(conv1): Conv1d(6, 6, kernel_size=(7,), stride=(1,), padding=(3,), bias=False)
(bn2): BatchNorm1d(6, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(conv2): Conv1d(6, 6, kernel_size=(7,), stride=(1,), padding=(3,), bias=False)
(residualConv): Conv1d(6, 6, kernel_size=(7,), stride=(1,), padding=(3,), bias=False)
)
)
(abpFc): Linear(in_features=2814, out_features=32, bias=True)
(fullLinear1): Linear(in_features=32, out_features=16, bias=True)
(fullLinear2): Linear(in_features=16, out_features=1, bias=True)
(sigmoid): Sigmoid()
)
Training Loop
100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:52<00:00, 3.96it/s] 100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:07<00:00, 4.95it/s]
[2024-05-01 05:52:15.939974] Completed epoch 0 with training loss 0.54845905, validation loss 0.59052509 Validation loss improved to 0.59052509. Model saved.
100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:52<00:00, 4.01it/s] 100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.15it/s]
[2024-05-01 05:53:14.869796] Completed epoch 1 with training loss 0.51682955, validation loss 0.65855759 No improvement in validation loss. 1 epochs without improvement.
100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:52<00:00, 4.00it/s] 100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.13it/s]
[2024-05-01 05:54:13.965398] Completed epoch 2 with training loss 0.51347697, validation loss 0.62033492 No improvement in validation loss. 2 epochs without improvement.
100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:51<00:00, 4.03it/s] 100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.12it/s]
[2024-05-01 05:55:12.682862] Completed epoch 3 with training loss 0.50908750, validation loss 0.55703592 Validation loss improved to 0.55703592. Model saved.
100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:52<00:00, 3.99it/s] 100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.14it/s]
[2024-05-01 05:56:11.883987] Completed epoch 4 with training loss 0.50609565, validation loss 0.55105549 Validation loss improved to 0.55105549. Model saved.
100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:52<00:00, 4.00it/s] 100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.13it/s]
[2024-05-01 05:57:10.977086] Completed epoch 5 with training loss 0.50368357, validation loss 0.62956876 No improvement in validation loss. 1 epochs without improvement.
100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:51<00:00, 4.02it/s] 100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.15it/s]
[2024-05-01 05:58:09.789318] Completed epoch 6 with training loss 0.50122613, validation loss 0.58714724 No improvement in validation loss. 2 epochs without improvement.
100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:52<00:00, 4.01it/s] 100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.13it/s]
[2024-05-01 05:59:08.739627] Completed epoch 7 with training loss 0.50275570, validation loss 0.55850273 No improvement in validation loss. 3 epochs without improvement.
100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:52<00:00, 4.00it/s] 100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.14it/s]
[2024-05-01 06:00:07.827975] Completed epoch 8 with training loss 0.49952757, validation loss 0.57284427 No improvement in validation loss. 4 epochs without improvement.
100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:52<00:00, 4.00it/s] 100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.11it/s]
[2024-05-01 06:01:06.902822] Completed epoch 9 with training loss 0.49783376, validation loss 0.59470636 No improvement in validation loss. 5 epochs without improvement.
100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:52<00:00, 4.01it/s] 100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.12it/s]
[2024-05-01 06:02:05.837565] Completed epoch 10 with training loss 0.49747616, validation loss 0.57383054 No improvement in validation loss. 6 epochs without improvement.
100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:52<00:00, 4.01it/s] 100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.15it/s]
[2024-05-01 06:03:04.819358] Completed epoch 11 with training loss 0.49439850, validation loss 0.55042309 Validation loss improved to 0.55042309. Model saved.
100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:52<00:00, 4.02it/s] 100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.16it/s]
[2024-05-01 06:04:03.688742] Completed epoch 12 with training loss 0.49488166, validation loss 0.57511932 No improvement in validation loss. 1 epochs without improvement.
100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:51<00:00, 4.03it/s] 100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.14it/s]
[2024-05-01 06:05:02.369610] Completed epoch 13 with training loss 0.49084646, validation loss 0.54866123 Validation loss improved to 0.54866123. Model saved.
100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:52<00:00, 4.01it/s] 100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.16it/s]
[2024-05-01 06:06:01.379038] Completed epoch 14 with training loss 0.49175668, validation loss 0.55878216 No improvement in validation loss. 1 epochs without improvement.
100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:52<00:00, 4.02it/s] 100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.15it/s]
[2024-05-01 06:07:00.245122] Completed epoch 15 with training loss 0.48997128, validation loss 0.55655724 No improvement in validation loss. 2 epochs without improvement.
100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:52<00:00, 4.01it/s] 100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.15it/s]
[2024-05-01 06:07:59.254088] Completed epoch 16 with training loss 0.48890623, validation loss 0.56272739 No improvement in validation loss. 3 epochs without improvement.
100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:52<00:00, 4.01it/s] 100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.17it/s]
[2024-05-01 06:08:58.201056] Completed epoch 17 with training loss 0.48559666, validation loss 0.56365335 No improvement in validation loss. 4 epochs without improvement.
100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:51<00:00, 4.03it/s] 100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.14it/s]
[2024-05-01 06:09:56.901396] Completed epoch 18 with training loss 0.48504612, validation loss 0.56514549 No improvement in validation loss. 5 epochs without improvement.
100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:52<00:00, 4.01it/s] 100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.15it/s]
[2024-05-01 06:10:55.858487] Completed epoch 19 with training loss 0.48484984, validation loss 0.54416662 Validation loss improved to 0.54416662. Model saved.
100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:52<00:00, 4.00it/s] 100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.12it/s]
[2024-05-01 06:11:54.952033] Completed epoch 20 with training loss 0.48363432, validation loss 0.54701400 No improvement in validation loss. 1 epochs without improvement.
100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:51<00:00, 4.03it/s] 100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.14it/s]
[2024-05-01 06:12:53.694357] Completed epoch 21 with training loss 0.48158345, validation loss 0.55995244 No improvement in validation loss. 2 epochs without improvement.
100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:52<00:00, 4.00it/s] 100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.10it/s]
[2024-05-01 06:13:52.836525] Completed epoch 22 with training loss 0.48322493, validation loss 0.57864243 No improvement in validation loss. 3 epochs without improvement.
100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:52<00:00, 4.01it/s] 100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.15it/s]
[2024-05-01 06:14:51.849248] Completed epoch 23 with training loss 0.48168632, validation loss 0.56460971 No improvement in validation loss. 4 epochs without improvement.
100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:51<00:00, 4.03it/s] 100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.11it/s]
[2024-05-01 06:15:50.600099] Completed epoch 24 with training loss 0.47952753, validation loss 0.55760300 No improvement in validation loss. 5 epochs without improvement.
100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:52<00:00, 4.01it/s] 100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.15it/s]
[2024-05-01 06:16:49.509678] Completed epoch 25 with training loss 0.48034054, validation loss 0.56507963 No improvement in validation loss. 6 epochs without improvement.
100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:52<00:00, 4.01it/s] 100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.08it/s]
[2024-05-01 06:17:48.594314] Completed epoch 26 with training loss 0.47821549, validation loss 0.54509956 No improvement in validation loss. 7 epochs without improvement.
100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:51<00:00, 4.04it/s] 100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.16it/s]
[2024-05-01 06:18:47.205087] Completed epoch 27 with training loss 0.47935650, validation loss 0.57744205 No improvement in validation loss. 8 epochs without improvement.
100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:52<00:00, 4.01it/s] 100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.04it/s]
[2024-05-01 06:19:46.323120] Completed epoch 28 with training loss 0.47750622, validation loss 0.54066294 Validation loss improved to 0.54066294. Model saved.
100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:52<00:00, 4.01it/s] 100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.15it/s]
[2024-05-01 06:20:45.363493] Completed epoch 29 with training loss 0.47652009, validation loss 0.54733419 No improvement in validation loss. 1 epochs without improvement.
100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:51<00:00, 4.03it/s] 100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.08it/s]
[2024-05-01 06:21:44.104325] Completed epoch 30 with training loss 0.47453353, validation loss 0.58113801 No improvement in validation loss. 2 epochs without improvement.
100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:52<00:00, 4.01it/s] 100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.16it/s]
[2024-05-01 06:22:43.089748] Completed epoch 31 with training loss 0.47749081, validation loss 0.54751712 No improvement in validation loss. 3 epochs without improvement.
100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:52<00:00, 4.00it/s] 100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.17it/s]
[2024-05-01 06:23:42.092817] Completed epoch 32 with training loss 0.47334585, validation loss 0.55001366 No improvement in validation loss. 4 epochs without improvement.
100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:52<00:00, 4.02it/s] 100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.15it/s]
[2024-05-01 06:24:40.943000] Completed epoch 33 with training loss 0.47331816, validation loss 0.55479157 No improvement in validation loss. 5 epochs without improvement.
100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:52<00:00, 4.01it/s] 100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.14it/s]
[2024-05-01 06:25:39.879635] Completed epoch 34 with training loss 0.47258097, validation loss 0.54148060 No improvement in validation loss. 6 epochs without improvement.
100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:52<00:00, 4.02it/s] 100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.16it/s]
[2024-05-01 06:26:38.735940] Completed epoch 35 with training loss 0.47086498, validation loss 0.54867750 No improvement in validation loss. 7 epochs without improvement.
100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:51<00:00, 4.03it/s] 100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.16it/s]
[2024-05-01 06:27:37.474818] Completed epoch 36 with training loss 0.47133112, validation loss 0.59381622 No improvement in validation loss. 8 epochs without improvement.
100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:52<00:00, 4.00it/s] 100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.15it/s]
[2024-05-01 06:28:36.592718] Completed epoch 37 with training loss 0.46956050, validation loss 0.55694813 No improvement in validation loss. 9 epochs without improvement.
100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:51<00:00, 4.02it/s] 100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.15it/s]
[2024-05-01 06:29:35.410165] Completed epoch 38 with training loss 0.46869338, validation loss 0.56618899 No improvement in validation loss. 10 epochs without improvement.
100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:52<00:00, 3.99it/s] 100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.17it/s]
[2024-05-01 06:30:34.549959] Completed epoch 39 with training loss 0.46828377, validation loss 0.54067415 No improvement in validation loss. 11 epochs without improvement.
100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:52<00:00, 4.00it/s] 100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.13it/s]
[2024-05-01 06:31:33.626364] Completed epoch 40 with training loss 0.46593815, validation loss 0.53823632 Validation loss improved to 0.53823632. Model saved.
100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:51<00:00, 4.03it/s] 100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.14it/s]
[2024-05-01 06:32:32.372382] Completed epoch 41 with training loss 0.46646926, validation loss 0.58251399 No improvement in validation loss. 1 epochs without improvement.
100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:52<00:00, 4.02it/s] 100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.14it/s]
[2024-05-01 06:33:31.225274] Completed epoch 42 with training loss 0.46279860, validation loss 0.53783387 Validation loss improved to 0.53783387. Model saved.
100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:52<00:00, 4.00it/s] 100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.13it/s]
[2024-05-01 06:34:30.365500] Completed epoch 43 with training loss 0.46181706, validation loss 0.57645744 No improvement in validation loss. 1 epochs without improvement.
100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:51<00:00, 4.03it/s] 100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.13it/s]
[2024-05-01 06:35:29.038430] Completed epoch 44 with training loss 0.46022037, validation loss 0.61514252 No improvement in validation loss. 2 epochs without improvement.
100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:51<00:00, 4.03it/s] 100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.14it/s]
[2024-05-01 06:36:27.686253] Completed epoch 45 with training loss 0.45952812, validation loss 0.55357546 No improvement in validation loss. 3 epochs without improvement.
100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:51<00:00, 4.04it/s] 100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.15it/s]
[2024-05-01 06:37:26.237294] Completed epoch 46 with training loss 0.45959979, validation loss 0.56573421 No improvement in validation loss. 4 epochs without improvement.
100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:51<00:00, 4.03it/s] 100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.14it/s]
[2024-05-01 06:38:24.946409] Completed epoch 47 with training loss 0.46286428, validation loss 0.56414813 No improvement in validation loss. 5 epochs without improvement.
100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:51<00:00, 4.02it/s] 100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.14it/s]
[2024-05-01 06:39:23.784364] Completed epoch 48 with training loss 0.45756015, validation loss 0.59026152 No improvement in validation loss. 6 epochs without improvement.
100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:52<00:00, 4.01it/s] 100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.13it/s]
[2024-05-01 06:40:22.709512] Completed epoch 49 with training loss 0.45804489, validation loss 0.54794979 No improvement in validation loss. 7 epochs without improvement.
100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:51<00:00, 4.03it/s] 100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.16it/s]
[2024-05-01 06:41:21.370362] Completed epoch 50 with training loss 0.45386106, validation loss 0.56097811 No improvement in validation loss. 8 epochs without improvement.
100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:51<00:00, 4.02it/s] 100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.13it/s]
[2024-05-01 06:42:20.165502] Completed epoch 51 with training loss 0.45701405, validation loss 0.60055518 No improvement in validation loss. 9 epochs without improvement.
100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:51<00:00, 4.05it/s] 100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.15it/s]
[2024-05-01 06:43:18.627322] Completed epoch 52 with training loss 0.45644122, validation loss 0.59112924 No improvement in validation loss. 10 epochs without improvement.
100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:51<00:00, 4.02it/s] 100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.13it/s]
[2024-05-01 06:44:17.415546] Completed epoch 53 with training loss 0.45395449, validation loss 0.61414361 No improvement in validation loss. 11 epochs without improvement.
100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:51<00:00, 4.04it/s] 100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.14it/s]
[2024-05-01 06:45:16.043160] Completed epoch 54 with training loss 0.45308992, validation loss 0.54727173 No improvement in validation loss. 12 epochs without improvement.
100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:51<00:00, 4.02it/s] 100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.13it/s]
[2024-05-01 06:46:14.848703] Completed epoch 55 with training loss 0.45197782, validation loss 0.57491523 No improvement in validation loss. 13 epochs without improvement.
100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:51<00:00, 4.03it/s] 100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.13it/s]
[2024-05-01 06:47:13.523378] Completed epoch 56 with training loss 0.45006117, validation loss 0.58290470 No improvement in validation loss. 14 epochs without improvement.
100%|█████████████████████████████████████████████████████████████████████████████████████| 209/209 [00:51<00:00, 4.03it/s] 100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.15it/s]
[2024-05-01 06:48:12.263781] Completed epoch 57 with training loss 0.45042974, validation loss 0.55502719 No improvement in validation loss. 15 epochs without improvement. Early stopping due to no improvement in validation loss. Plot Validation and Loss Values from Training Epoch with best Validation Loss: 42, 0.5378
Generate AUROC/AUPRC for Each Intermediate Model Intermediate Model: ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0000.model AUROC/AUPRC on Validation Data
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.73it/s]
Loss: 0.5859471900122506 AUROC: 0.8035949922324773 AUPRC: 0.7038041366388864 Sensitivity: 0.7161290322580646 Specificity: 0.7407932011331445 Threshold: 0.25 AUROC/AUPRC on Test Data
100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:21<00:00, 4.94it/s]
Loss: 0.6083608744841702 AUROC: 0.7939504925486234 AUPRC: 0.7157216571132787 Sensitivity: 0.716781796017879 Specificity: 0.7313606710158435 Threshold: 0.24 Intermediate Model: ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0001.model AUROC/AUPRC on Validation Data
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.06it/s]
Loss: 0.6574152963502067 AUROC: 0.8078506808005117 AUPRC: 0.7093965725413414 Sensitivity: 0.7367741935483871 Specificity: 0.7315864022662889 Threshold: 0.15 AUROC/AUPRC on Test Data
100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:21<00:00, 5.02it/s]
Loss: 0.6902545052316954 AUROC: 0.7967412321876445 AUPRC: 0.7205366976873793 Sensitivity: 0.7326290125965055 Specificity: 0.7134203168685928 Threshold: 0.14 Intermediate Model: ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0002.model AUROC/AUPRC on Validation Data
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.09it/s]
Loss: 0.6226890529905047 AUROC: 0.8079119071552592 AUPRC: 0.7081702299849927 Sensitivity: 0.7303225806451613 Specificity: 0.740084985835694 Threshold: 0.18 AUROC/AUPRC on Test Data
100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:21<00:00, 4.88it/s]
Loss: 0.6511145466060009 AUROC: 0.7967217767726392 AUPRC: 0.7199543664234093 Sensitivity: 0.7245022348638764 Specificity: 0.7215750232991612 Threshold: 0.17 Intermediate Model: ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0003.model AUROC/AUPRC on Validation Data
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:07<00:00, 4.81it/s]
Loss: 0.5725380327020373 AUROC: 0.8070858082792651 AUPRC: 0.7058901550885577 Sensitivity: 0.7290322580645161 Specificity: 0.7415014164305949 Threshold: 0.28 AUROC/AUPRC on Test Data
100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:21<00:00, 5.04it/s]
Loss: 0.5797299520587021 AUROC: 0.7952703365417569 AUPRC: 0.717508215000471 Sensitivity: 0.7216578626574564 Specificity: 0.7271668219944082 Threshold: 0.27 Intermediate Model: ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0004.model AUROC/AUPRC on Validation Data
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.12it/s]
Loss: 0.5600509328501565 AUROC: 0.8067979530293337 AUPRC: 0.7051440989632735 Sensitivity: 0.7354838709677419 Specificity: 0.7330028328611898 Threshold: 0.29 AUROC/AUPRC on Test Data
100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:20<00:00, 5.06it/s]
Loss: 0.5746631549214417 AUROC: 0.7947537029666526 AUPRC: 0.7167484861064243 Sensitivity: 0.728565623730191 Specificity: 0.717381174277726 Threshold: 0.28 Intermediate Model: ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0005.model AUROC/AUPRC on Validation Data
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.13it/s]
Loss: 0.6199249812534877 AUROC: 0.8086292607146122 AUPRC: 0.7081687466359474 Sensitivity: 0.7380645161290322 Specificity: 0.7280453257790368 Threshold: 0.17 AUROC/AUPRC on Test Data
100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:20<00:00, 5.05it/s]
Loss: 0.6545365085860468 AUROC: 0.7964395549131218 AUPRC: 0.7179992459583092 Sensitivity: 0.7180008126777733 Specificity: 0.7374184529356943 Threshold: 0.17 Intermediate Model: ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0006.model AUROC/AUPRC on Validation Data
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.09it/s]
Loss: 0.5778036177158355 AUROC: 0.8070519967102258 AUPRC: 0.7064476366545598 Sensitivity: 0.7406451612903225 Specificity: 0.7344192634560907 Threshold: 0.23 AUROC/AUPRC on Test Data
100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:21<00:00, 5.03it/s]
Loss: 0.6084165859897182 AUROC: 0.7949990968143107 AUPRC: 0.7162418904878575 Sensitivity: 0.7273466070702966 Specificity: 0.7197110904007455 Threshold: 0.22 Intermediate Model: ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0007.model AUROC/AUPRC on Validation Data
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.01it/s]
Loss: 0.562591245344707 AUROC: 0.8059298181485881 AUPRC: 0.7066158010613205 Sensitivity: 0.7316129032258064 Specificity: 0.7443342776203966 Threshold: 0.29 AUROC/AUPRC on Test Data
100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:21<00:00, 5.02it/s]
Loss: 0.5772552568957491 AUROC: 0.7934264744364368 AUPRC: 0.7151381454828599 Sensitivity: 0.7139374238114587 Specificity: 0.7329916123019571 Threshold: 0.28 Intermediate Model: ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0008.model AUROC/AUPRC on Validation Data
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.09it/s]
Loss: 0.564673832484654 AUROC: 0.805629169331993 AUPRC: 0.7070136066002808 Sensitivity: 0.743225806451613 Specificity: 0.7330028328611898 Threshold: 0.28 AUROC/AUPRC on Test Data
100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:21<00:00, 5.03it/s]
Loss: 0.5854992335135082 AUROC: 0.7919913180565564 AUPRC: 0.7106699701122675 Sensitivity: 0.7289719626168224 Specificity: 0.7143522833178005 Threshold: 0.27 Intermediate Model: ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0009.model AUROC/AUPRC on Validation Data
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.10it/s]
Loss: 0.6133006725992475 AUROC: 0.8072000365530476 AUPRC: 0.7097292973082021 Sensitivity: 0.7406451612903225 Specificity: 0.7351274787535411 Threshold: 0.21 AUROC/AUPRC on Test Data
100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:21<00:00, 5.03it/s]
Loss: 0.6247424463618476 AUROC: 0.7932481568006096 AUPRC: 0.7127362663016106 Sensitivity: 0.7273466070702966 Specificity: 0.7183131407269339 Threshold: 0.2 Intermediate Model: ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0010.model AUROC/AUPRC on Validation Data
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.10it/s]
Loss: 0.5694470907960619 AUROC: 0.8063684547199124 AUPRC: 0.7088218248978105 Sensitivity: 0.7406451612903225 Specificity: 0.7330028328611898 Threshold: 0.25 AUROC/AUPRC on Test Data
100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:21<00:00, 5.02it/s]
Loss: 0.594383901020266 AUROC: 0.7921538725459194 AUPRC: 0.7108029858030867 Sensitivity: 0.7184071515644047 Specificity: 0.7287977632805219 Threshold: 0.25 Intermediate Model: ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0011.model AUROC/AUPRC on Validation Data
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.09it/s]
Loss: 0.5581223368644714 AUROC: 0.8070538243626063 AUPRC: 0.7062225913641743 Sensitivity: 0.7341935483870968 Specificity: 0.7407932011331445 Threshold: 0.3 AUROC/AUPRC on Test Data
100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:21<00:00, 5.02it/s]
Loss: 0.5705225805066666 AUROC: 0.7920145130768791 AUPRC: 0.7090080756646415 Sensitivity: 0.7228768793173507 Specificity: 0.7220410065237651 Threshold: 0.29 Intermediate Model: ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0012.model AUROC/AUPRC on Validation Data
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.10it/s]
Loss: 0.5769370462213244 AUROC: 0.8073690943982454 AUPRC: 0.7085046652374707 Sensitivity: 0.7393548387096774 Specificity: 0.7386685552407932 Threshold: 0.25 AUROC/AUPRC on Test Data
100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:21<00:00, 5.04it/s]
Loss: 0.5954774980837444 AUROC: 0.7910610557312907 AUPRC: 0.7068395544474011 Sensitivity: 0.7139374238114587 Specificity: 0.7297297297297297 Threshold: 0.25 Intermediate Model: ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0013.model AUROC/AUPRC on Validation Data
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.11it/s]
Loss: 0.5498952754906246 AUROC: 0.80374577355387 AUPRC: 0.7053207258327664 Sensitivity: 0.7393548387096774 Specificity: 0.7358356940509915 Threshold: 0.31 AUROC/AUPRC on Test Data
100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:20<00:00, 5.06it/s]
Loss: 0.570091428902914 AUROC: 0.7883626701425746 AUPRC: 0.7050630023487928 Sensitivity: 0.712312068264933 Specificity: 0.7257688723205965 Threshold: 0.31 Intermediate Model: ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0014.model AUROC/AUPRC on Validation Data
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.10it/s]
Loss: 0.5611233958176204 AUROC: 0.8049977154345243 AUPRC: 0.7047176175952845 Sensitivity: 0.7380645161290322 Specificity: 0.7372521246458924 Threshold: 0.28 AUROC/AUPRC on Test Data
100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:21<00:00, 5.04it/s]
Loss: 0.5802073644579582 AUROC: 0.7883992614705528 AUPRC: 0.7029214581590661 Sensitivity: 0.7127184071515644 Specificity: 0.7215750232991612 Threshold: 0.28 Intermediate Model: ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0015.model AUROC/AUPRC on Validation Data
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.08it/s]
Loss: 0.5585595897265843 AUROC: 0.8030256785159463 AUPRC: 0.7025061807781666 Sensitivity: 0.727741935483871 Specificity: 0.7450424929178471 Threshold: 0.29 AUROC/AUPRC on Test Data
100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:22<00:00, 4.76it/s]
Loss: 0.5811271973938312 AUROC: 0.7860195943957801 AUPRC: 0.6982504170101814 Sensitivity: 0.7082486793986185 Specificity: 0.72856477166822 Threshold: 0.29 Intermediate Model: ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0016.model AUROC/AUPRC on Validation Data
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:07<00:00, 4.93it/s]
Loss: 0.5636099057538169 AUROC: 0.8058192451795668 AUPRC: 0.7056834488288954 Sensitivity: 0.7393548387096774 Specificity: 0.7365439093484419 Threshold: 0.27 AUROC/AUPRC on Test Data
100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:21<00:00, 4.96it/s]
Loss: 0.5838960510379864 AUROC: 0.7880805429566096 AUPRC: 0.7006285068504388 Sensitivity: 0.7143437626980902 Specificity: 0.7253028890959925 Threshold: 0.27 Intermediate Model: ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0017.model AUROC/AUPRC on Validation Data
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.02it/s]
Loss: 0.5634740301540919 AUROC: 0.8058512290962259 AUPRC: 0.7054162528617351 Sensitivity: 0.7367741935483871 Specificity: 0.7422096317280453 Threshold: 0.26 AUROC/AUPRC on Test Data
100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:21<00:00, 5.01it/s]
Loss: 0.5902659772139676 AUROC: 0.788095548714655 AUPRC: 0.6998786104902175 Sensitivity: 0.7143437626980902 Specificity: 0.7267008387698043 Threshold: 0.26 Intermediate Model: ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0018.model AUROC/AUPRC on Validation Data
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.06it/s]
Loss: 0.5628924182483128 AUROC: 0.8062112766151878 AUPRC: 0.7042849880875 Sensitivity: 0.7419354838709677 Specificity: 0.7337110481586402 Threshold: 0.25 AUROC/AUPRC on Test Data
100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:21<00:00, 4.98it/s]
Loss: 0.592963512635456 AUROC: 0.7878858467962281 AUPRC: 0.6982225982157145 Sensitivity: 0.7204388459975619 Specificity: 0.7176141658900279 Threshold: 0.25 Intermediate Model: ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0019.model AUROC/AUPRC on Validation Data
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.00it/s]
Loss: 0.5464371119226729 AUROC: 0.7986968838526912 AUPRC: 0.68996813509794 Sensitivity: 0.7406451612903225 Specificity: 0.7273371104815864 Threshold: 0.36 AUROC/AUPRC on Test Data
100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:21<00:00, 4.96it/s]
Loss: 0.5600495630840086 AUROC: 0.7821669015201921 AUPRC: 0.6853861735625737 Sensitivity: 0.7151564404713531 Specificity: 0.7103914259086673 Threshold: 0.36 Intermediate Model: ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0020.model AUROC/AUPRC on Validation Data
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.08it/s]
Loss: 0.5495232633181981 AUROC: 0.803151786530202 AUPRC: 0.6989712330591584 Sensitivity: 0.7329032258064516 Specificity: 0.7415014164305949 Threshold: 0.31 AUROC/AUPRC on Test Data
100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:21<00:00, 4.93it/s]
Loss: 0.567583792052179 AUROC: 0.7850250013917013 AUPRC: 0.6921835864926364 Sensitivity: 0.724095895977245 Specificity: 0.706663560111836 Threshold: 0.3 Intermediate Model: ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0021.model AUROC/AUPRC on Validation Data
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.03it/s]
Loss: 0.5650256276130676 AUROC: 0.8047683450607694 AUPRC: 0.7002016638675831 Sensitivity: 0.7406451612903225 Specificity: 0.7337110481586402 Threshold: 0.27 AUROC/AUPRC on Test Data
100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:21<00:00, 4.95it/s]
Loss: 0.5806394601767918 AUROC: 0.7850549182342398 AUPRC: 0.6908906882457168 Sensitivity: 0.7184071515644047 Specificity: 0.717148182665424 Threshold: 0.27 Intermediate Model: ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0022.model AUROC/AUPRC on Validation Data
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.02it/s]
Loss: 0.5743089182036264 AUROC: 0.8080544640409394 AUPRC: 0.7035868991249145 Sensitivity: 0.7445161290322581 Specificity: 0.7351274787535411 Threshold: 0.22 AUROC/AUPRC on Test Data
100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:21<00:00, 4.97it/s]
Loss: 0.6047967499438321 AUROC: 0.7879760233548292 AUPRC: 0.6957865412328825 Sensitivity: 0.7208451848841935 Specificity: 0.7190121155638397 Threshold: 0.22 Intermediate Model: ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0023.model AUROC/AUPRC on Validation Data
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:07<00:00, 4.89it/s]
Loss: 0.5672739991119929 AUROC: 0.8063474367175362 AUPRC: 0.7021889132817944 Sensitivity: 0.7458064516129033 Specificity: 0.7322946175637394 Threshold: 0.23 AUROC/AUPRC on Test Data
100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:21<00:00, 4.94it/s]
Loss: 0.5984816556831576 AUROC: 0.7864378621500061 AUPRC: 0.6935190437345973 Sensitivity: 0.724095895977245 Specificity: 0.7103914259086673 Threshold: 0.23 Intermediate Model: ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0024.model AUROC/AUPRC on Validation Data
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.01it/s]
Loss: 0.553110134601593 AUROC: 0.8054496024856073 AUPRC: 0.6974905780082327 Sensitivity: 0.7367741935483871 Specificity: 0.7464589235127479 Threshold: 0.25 AUROC/AUPRC on Test Data
100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:21<00:00, 5.00it/s]
Loss: 0.585744377014772 AUROC: 0.7857363311271871 AUPRC: 0.6905640573334084 Sensitivity: 0.7163754571312475 Specificity: 0.7206430568499534 Threshold: 0.25 Intermediate Model: ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0025.model AUROC/AUPRC on Validation Data
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.03it/s]
Loss: 0.5576985061168671 AUROC: 0.8118989308233574 AUPRC: 0.7029533103893714 Sensitivity: 0.7458064516129033 Specificity: 0.7464589235127479 Threshold: 0.22 AUROC/AUPRC on Test Data
100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:21<00:00, 4.88it/s]
Loss: 0.5950879606435884 AUROC: 0.7910940968010564 AUPRC: 0.697151972063846 Sensitivity: 0.716781796017879 Specificity: 0.7248369058713886 Threshold: 0.22 Intermediate Model: ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0026.model AUROC/AUPRC on Validation Data
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.03it/s]
Loss: 0.5549319905894143 AUROC: 0.8081339669194919 AUPRC: 0.6974055493821754 Sensitivity: 0.7445161290322581 Specificity: 0.7386685552407932 Threshold: 0.26 AUROC/AUPRC on Test Data
100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:21<00:00, 4.98it/s]
Loss: 0.5776107519302728 AUROC: 0.7850932610229364 AUPRC: 0.68671659734642 Sensitivity: 0.7184071515644047 Specificity: 0.7166821994408201 Threshold: 0.26 Intermediate Model: ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0027.model AUROC/AUPRC on Validation Data
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:07<00:00, 5.00it/s]
Loss: 0.5719426946980612 AUROC: 0.8046248743488988 AUPRC: 0.6947635853432876 Sensitivity: 0.7445161290322581 Specificity: 0.7351274787535411 Threshold: 0.21 AUROC/AUPRC on Test Data
100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:22<00:00, 4.81it/s]
Loss: 0.6098847259890359 AUROC: 0.7813553598295573 AUPRC: 0.6836268871625606 Sensitivity: 0.7216578626574564 Specificity: 0.7031686859273066 Threshold: 0.21 Intermediate Model: ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0028.model AUROC/AUPRC on Validation Data
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:07<00:00, 4.92it/s]
Loss: 0.5487099630492074 AUROC: 0.8051594626702 AUPRC: 0.6937846237361699 Sensitivity: 0.7380645161290322 Specificity: 0.7450424929178471 Threshold: 0.28 AUROC/AUPRC on Test Data
100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:22<00:00, 4.79it/s]
Loss: 0.5722965238229284 AUROC: 0.7817214151196693 AUPRC: 0.6811380012342292 Sensitivity: 0.7102803738317757 Specificity: 0.717381174277726 Threshold: 0.28 Intermediate Model: ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0029.model AUROC/AUPRC on Validation Data
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:07<00:00, 4.93it/s]
Loss: 0.5505596595151084 AUROC: 0.809671022571507 AUPRC: 0.6983587729207007 Sensitivity: 0.7509677419354839 Specificity: 0.7365439093484419 Threshold: 0.24 AUROC/AUPRC on Test Data
100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:20<00:00, 5.06it/s]
Loss: 0.5812201449331248 AUROC: 0.7848498079831012 AUPRC: 0.6876830968719401 Sensitivity: 0.7257212515237709 Specificity: 0.7087604846225536 Threshold: 0.24 Intermediate Model: ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0030.model AUROC/AUPRC on Validation Data
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.10it/s]
Loss: 0.5660754327263151 AUROC: 0.8061262907794939 AUPRC: 0.6966148572809036 Sensitivity: 0.7458064516129033 Specificity: 0.7436260623229461 Threshold: 0.21 AUROC/AUPRC on Test Data
100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:21<00:00, 5.04it/s]
Loss: 0.6084756879311688 AUROC: 0.7828351547893646 AUPRC: 0.6891518846583943 Sensitivity: 0.7135310849248273 Specificity: 0.71784715750233 Threshold: 0.21 Intermediate Model: ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0031.model AUROC/AUPRC on Validation Data
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.09it/s]
Loss: 0.5451005203383309 AUROC: 0.8120515397971305 AUPRC: 0.7017753683626013 Sensitivity: 0.7470967741935484 Specificity: 0.740084985835694 Threshold: 0.24 AUROC/AUPRC on Test Data
100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:21<00:00, 5.02it/s]
Loss: 0.5791899400218478 AUROC: 0.7882354288882333 AUPRC: 0.6927611487329255 Sensitivity: 0.7204388459975619 Specificity: 0.7213420316868593 Threshold: 0.24 Intermediate Model: ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0032.model AUROC/AUPRC on Validation Data
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.11it/s]
Loss: 0.5467198865754264 AUROC: 0.811010691766426 AUPRC: 0.6948754131554438 Sensitivity: 0.7445161290322581 Specificity: 0.7450424929178471 Threshold: 0.24 AUROC/AUPRC on Test Data
100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:21<00:00, 5.04it/s]
Loss: 0.5831167169337003 AUROC: 0.7863374135109763 AUPRC: 0.6860904570947859 Sensitivity: 0.7135310849248273 Specificity: 0.7287977632805219 Threshold: 0.24 Intermediate Model: ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0033.model AUROC/AUPRC on Validation Data
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.12it/s]
Loss: 0.5557353377342225 AUROC: 0.804773828017911 AUPRC: 0.6887814305584704 Sensitivity: 0.7329032258064516 Specificity: 0.7436260623229461 Threshold: 0.24 AUROC/AUPRC on Test Data
100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:20<00:00, 5.05it/s]
Loss: 0.5917413386533845 AUROC: 0.7779493841106726 AUPRC: 0.6751847707990541 Sensitivity: 0.7082486793986185 Specificity: 0.7176141658900279 Threshold: 0.24 Intermediate Model: ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0034.model AUROC/AUPRC on Validation Data
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.11it/s]
Loss: 0.5426491243498666 AUROC: 0.7966709311888879 AUPRC: 0.6772665266566317 Sensitivity: 0.7341935483870968 Specificity: 0.7393767705382436 Threshold: 0.3 AUROC/AUPRC on Test Data
100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:20<00:00, 5.05it/s]
Loss: 0.5715162391370198 AUROC: 0.7684352601420937 AUPRC: 0.6596609927205713 Sensitivity: 0.7074360016253556 Specificity: 0.7024697110904008 Threshold: 0.3 Intermediate Model: ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0035.model AUROC/AUPRC on Validation Data
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.11it/s]
Loss: 0.5551485044615609 AUROC: 0.8119711230923878 AUPRC: 0.6929331013836794 Sensitivity: 0.7445161290322581 Specificity: 0.7563739376770539 Threshold: 0.24 AUROC/AUPRC on Test Data
100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:21<00:00, 5.04it/s]
Loss: 0.585233063349184 AUROC: 0.7835888509395214 AUPRC: 0.680153000707304 Sensitivity: 0.7257212515237709 Specificity: 0.7150512581547064 Threshold: 0.23 Intermediate Model: ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0036.model AUROC/AUPRC on Validation Data
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.07it/s]
Loss: 0.59342514531953 AUROC: 0.8145033354655944 AUPRC: 0.6985433361793169 Sensitivity: 0.7470967741935484 Specificity: 0.7429178470254958 Threshold: 0.17 AUROC/AUPRC on Test Data
100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:20<00:00, 5.05it/s]
Loss: 0.6376918984471627 AUROC: 0.7855488774935593 AUPRC: 0.6870615873265645 Sensitivity: 0.7147501015847216 Specificity: 0.7220410065237651 Threshold: 0.17 Intermediate Model: ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0037.model AUROC/AUPRC on Validation Data
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.11it/s]
Loss: 0.5671910762786865 AUROC: 0.8074796673672667 AUPRC: 0.6865932465524954 Sensitivity: 0.7445161290322581 Specificity: 0.7337110481586402 Threshold: 0.22 AUROC/AUPRC on Test Data
100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:21<00:00, 5.04it/s]
Loss: 0.598028153040499 AUROC: 0.7777144990273238 AUPRC: 0.6721383858812077 Sensitivity: 0.7106867127184071 Specificity: 0.7096924510717614 Threshold: 0.22 Intermediate Model: ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0038.model AUROC/AUPRC on Validation Data
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.10it/s]
Loss: 0.5699493067605155 AUROC: 0.8062286393128028 AUPRC: 0.6898170992994509 Sensitivity: 0.7329032258064516 Specificity: 0.7436260623229461 Threshold: 0.21 AUROC/AUPRC on Test Data
100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:21<00:00, 5.04it/s]
Loss: 0.6115994709280302 AUROC: 0.776156219692629 AUPRC: 0.673233595731711 Sensitivity: 0.7025599349857782 Specificity: 0.7141192917054986 Threshold: 0.21 Intermediate Model: ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0039.model AUROC/AUPRC on Validation Data
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.11it/s]
Loss: 0.5418225186211723 AUROC: 0.8052608973773188 AUPRC: 0.6875161601611246 Sensitivity: 0.7341935483870968 Specificity: 0.7429178470254958 Threshold: 0.28 AUROC/AUPRC on Test Data
100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:21<00:00, 5.05it/s]
Loss: 0.5686275393895384 AUROC: 0.7770466244523608 AUPRC: 0.6732609200396472 Sensitivity: 0.712312068264933 Specificity: 0.7136533084808947 Threshold: 0.28 Intermediate Model: ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0040.model AUROC/AUPRC on Validation Data
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.03it/s]
Loss: 0.5294744738510677 AUROC: 0.802695787261263 AUPRC: 0.6828664971574568 Sensitivity: 0.7445161290322581 Specificity: 0.7351274787535411 Threshold: 0.3 AUROC/AUPRC on Test Data
100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:20<00:00, 5.05it/s]
Loss: 0.5603052057185263 AUROC: 0.7757717977333637 AUPRC: 0.6705684926405954 Sensitivity: 0.7147501015847216 Specificity: 0.7047996272134203 Threshold: 0.3 Intermediate Model: ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0041.model AUROC/AUPRC on Validation Data
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.12it/s]
Loss: 0.5684827327728271 AUROC: 0.8008754454902678 AUPRC: 0.6811940670722644 Sensitivity: 0.7341935483870968 Specificity: 0.740084985835694 Threshold: 0.22 AUROC/AUPRC on Test Data
100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:20<00:00, 5.05it/s]
Loss: 0.606844001221207 AUROC: 0.7708991866784466 AUPRC: 0.6622585101888001 Sensitivity: 0.7086550182852499 Specificity: 0.7034016775396086 Threshold: 0.22 Intermediate Model: ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0042.model AUROC/AUPRC on Validation Data
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.12it/s]
Loss: 0.5387681143624442 AUROC: 0.7904989490998813 AUPRC: 0.6614565033215394 Sensitivity: 0.7225806451612903 Specificity: 0.7337110481586402 Threshold: 0.32 AUROC/AUPRC on Test Data
100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:20<00:00, 5.06it/s]
Loss: 0.5680583552369531 AUROC: 0.7625166010074024 AUPRC: 0.6472575073859037 Sensitivity: 0.7021535960991467 Specificity: 0.6982758620689655 Threshold: 0.32 Intermediate Model: ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0043.model AUROC/AUPRC on Validation Data
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.11it/s]
Loss: 0.5835100889205933 AUROC: 0.8109065155807365 AUPRC: 0.6968146093746534 Sensitivity: 0.7367741935483871 Specificity: 0.7478753541076487 Threshold: 0.19 AUROC/AUPRC on Test Data
100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:20<00:00, 5.07it/s]
Loss: 0.6212569905339547 AUROC: 0.7800931246930211 AUPRC: 0.6792467131004971 Sensitivity: 0.7049979683055668 Specificity: 0.7215750232991612 Threshold: 0.19 Intermediate Model: ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0044.model AUROC/AUPRC on Validation Data
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.03it/s]
Loss: 0.6088284913982663 AUROC: 0.8031947363611441 AUPRC: 0.6849703468101059 Sensitivity: 0.7264516129032258 Specificity: 0.7471671388101983 Threshold: 0.16 AUROC/AUPRC on Test Data
100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:21<00:00, 5.04it/s]
Loss: 0.6648801493476022 AUROC: 0.7713121527137415 AUPRC: 0.6650315564846779 Sensitivity: 0.7013409183258837 Specificity: 0.7103914259086673 Threshold: 0.16 Intermediate Model: ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0045.model AUROC/AUPRC on Validation Data
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.07it/s]
Loss: 0.5572233072349003 AUROC: 0.7952279996344696 AUPRC: 0.6715206105783889 Sensitivity: 0.7341935483870968 Specificity: 0.726628895184136 Threshold: 0.25 AUROC/AUPRC on Test Data
100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:21<00:00, 5.04it/s]
Loss: 0.5900252191525586 AUROC: 0.7656076451544372 AUPRC: 0.6565478338859068 Sensitivity: 0.6976838683462008 Specificity: 0.712022367194781 Threshold: 0.26 Intermediate Model: ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0046.model AUROC/AUPRC on Validation Data
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.08it/s]
Loss: 0.5716327769415719 AUROC: 0.7985077218313077 AUPRC: 0.6738721429128037 Sensitivity: 0.7354838709677419 Specificity: 0.7245042492917847 Threshold: 0.21 AUROC/AUPRC on Test Data
100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:21<00:00, 5.04it/s]
Loss: 0.6110387450119235 AUROC: 0.7673704193621805 AUPRC: 0.6558065879443377 Sensitivity: 0.6956521739130435 Specificity: 0.7129543336439889 Threshold: 0.22 Intermediate Model: ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0047.model AUROC/AUPRC on Validation Data
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.11it/s]
Loss: 0.5617766686848231 AUROC: 0.7944859727679795 AUPRC: 0.6725414733710708 Sensitivity: 0.7290322580645161 Specificity: 0.7252124645892352 Threshold: 0.23 AUROC/AUPRC on Test Data
100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:21<00:00, 5.04it/s]
Loss: 0.6017588793106798 AUROC: 0.7642914460930686 AUPRC: 0.6542888836266585 Sensitivity: 0.6980902072328322 Specificity: 0.7092264678471575 Threshold: 0.24 Intermediate Model: ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0048.model AUROC/AUPRC on Validation Data
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.11it/s]
Loss: 0.5835490499223982 AUROC: 0.805112857534497 AUPRC: 0.6893230412971221 Sensitivity: 0.7303225806451613 Specificity: 0.7407932011331445 Threshold: 0.18 AUROC/AUPRC on Test Data
100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:21<00:00, 5.00it/s]
Loss: 0.6333599045591535 AUROC: 0.7759493106439961 AUPRC: 0.6748884880709327 Sensitivity: 0.7119057293783015 Specificity: 0.7085274930102516 Threshold: 0.18 Intermediate Model: ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0049.model AUROC/AUPRC on Validation Data
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.06it/s]
Loss: 0.543922437940325 AUROC: 0.7914310518139451 AUPRC: 0.6670047380810928 Sensitivity: 0.7290322580645161 Specificity: 0.7237960339943342 Threshold: 0.27 AUROC/AUPRC on Test Data
100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:21<00:00, 5.02it/s]
Loss: 0.5811180154669959 AUROC: 0.763947402403875 AUPRC: 0.6531659544535098 Sensitivity: 0.7029662738724096 Specificity: 0.7027027027027027 Threshold: 0.28 Intermediate Model: ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0050.model AUROC/AUPRC on Validation Data
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.11it/s]
Loss: 0.5589352692876544 AUROC: 0.7957698985652929 AUPRC: 0.672134799645191 Sensitivity: 0.7329032258064516 Specificity: 0.7322946175637394 Threshold: 0.23 AUROC/AUPRC on Test Data
100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:20<00:00, 5.05it/s]
Loss: 0.5996047857235063 AUROC: 0.7668475373326219 AUPRC: 0.6564470666656923 Sensitivity: 0.7131247460381959 Specificity: 0.6971109040074557 Threshold: 0.23 Intermediate Model: ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0051.model AUROC/AUPRC on Validation Data
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.11it/s]
Loss: 0.5998892911842891 AUROC: 0.7972978159554052 AUPRC: 0.6761794106464618 Sensitivity: 0.7354838709677419 Specificity: 0.726628895184136 Threshold: 0.17 AUROC/AUPRC on Test Data
100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:20<00:00, 5.06it/s]
Loss: 0.6468638114209445 AUROC: 0.7676024642389592 AUPRC: 0.6596352366886208 Sensitivity: 0.6940268183665177 Specificity: 0.7157502329916123 Threshold: 0.18 Intermediate Model: ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0052.model AUROC/AUPRC on Validation Data
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.12it/s]
Loss: 0.5904938408306667 AUROC: 0.7943909348441925 AUPRC: 0.6769464555541472 Sensitivity: 0.7174193548387097 Specificity: 0.7358356940509915 Threshold: 0.2 AUROC/AUPRC on Test Data
100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:21<00:00, 5.01it/s]
Loss: 0.626921148232694 AUROC: 0.7653639554307212 AUPRC: 0.6570010644232122 Sensitivity: 0.7013409183258837 Specificity: 0.7024697110904008 Threshold: 0.2 Intermediate Model: ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0053.model AUROC/AUPRC on Validation Data
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.10it/s]
Loss: 0.6117460523332868 AUROC: 0.7999086173809741 AUPRC: 0.6830432028058967 Sensitivity: 0.7212903225806452 Specificity: 0.7415014164305949 Threshold: 0.16 AUROC/AUPRC on Test Data
100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:21<00:00, 5.05it/s]
Loss: 0.6636719661501219 AUROC: 0.769139678708259 AUPRC: 0.6642929080974225 Sensitivity: 0.6960585127996749 Specificity: 0.7157502329916123 Threshold: 0.16 Intermediate Model: ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0054.model AUROC/AUPRC on Validation Data
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.10it/s]
Loss: 0.5560744200434004 AUROC: 0.7962816412318376 AUPRC: 0.6759771134138498 Sensitivity: 0.7225806451612903 Specificity: 0.740084985835694 Threshold: 0.25 AUROC/AUPRC on Test Data
100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:20<00:00, 5.05it/s]
Loss: 0.5885736796091188 AUROC: 0.7671831550756574 AUPRC: 0.6586858145320005 Sensitivity: 0.699715562779358 Specificity: 0.712022367194781 Threshold: 0.25 Intermediate Model: ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0055.model AUROC/AUPRC on Validation Data
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.12it/s]
Loss: 0.575568094423839 AUROC: 0.7938271040848032 AUPRC: 0.675297412476766 Sensitivity: 0.727741935483871 Specificity: 0.7273371104815864 Threshold: 0.21 AUROC/AUPRC on Test Data
100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:20<00:00, 5.05it/s]
Loss: 0.6161271129857819 AUROC: 0.7651117924240709 AUPRC: 0.6558585555387731 Sensitivity: 0.7029662738724096 Specificity: 0.6978098788443616 Threshold: 0.21 Intermediate Model: ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0056.model AUROC/AUPRC on Validation Data
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.09it/s]
Loss: 0.5932718532426017 AUROC: 0.7941898930823358 AUPRC: 0.6761714026733244 Sensitivity: 0.7316129032258064 Specificity: 0.7259206798866855 Threshold: 0.19 AUROC/AUPRC on Test Data
100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:21<00:00, 5.02it/s]
Loss: 0.6316090141827205 AUROC: 0.7631830081423042 AUPRC: 0.6537447172600768 Sensitivity: 0.7033726127590411 Specificity: 0.6943150046598322 Threshold: 0.19 Intermediate Model: ./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0057.model AUROC/AUPRC on Validation Data
100%|███████████████████████████████████████████████████████████████████████████████████████| 35/35 [00:06<00:00, 5.11it/s]
Loss: 0.5482843109539577 AUROC: 0.7936260623229462 AUPRC: 0.6725973359170988 Sensitivity: 0.7290322580645161 Specificity: 0.7372521246458924 Threshold: 0.25 AUROC/AUPRC on Test Data
100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:20<00:00, 5.06it/s]
Loss: 0.5915976748713907 AUROC: 0.7668495728139971 AUPRC: 0.6587494508486212 Sensitivity: 0.7017472572125152 Specificity: 0.7050326188257223 Threshold: 0.25 Plot AUROC/AUPRC for Each Intermediate Model Epoch with best Validation Loss: 42, 0.5378 Epoch with best model Test AUROC: 1, 0.7967
AUROC/AUPRC Plots - Best Model Based on Validation Loss
Epoch with best Validation Loss: 42, 0.5378
Best Model Based on Validation Loss:
./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0042.model
Generate Stats Based on Test Data
100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:20<00:00, 5.05it/s]
Loss: 0.5680583552369531 AUROC: 0.7625166010074024 AUPRC: 0.6472575073859037 Sensitivity: 0.7021535960991467 Specificity: 0.6982758620689655 Threshold: 0.32
best_model_val_test_auroc: 0.7625166010074024
best_model_val_test_auprc: 0.6472575073859037
AUROC/AUPRC Plots - Best Model Based on Model AUROC
Epoch with best model Test AUROC: 1, 0.7967
Best Model Based on Model AUROC:
./vitaldb_cache/models/ABP_12_RESIDUAL_BLOCKS_64_BATCH_SIZE_0.0001_LEARNING_RATE_003_MINS__ALL_MAX_CASES_0001.model
Generate Stats Based on Test Data
100%|█████████████████████████████████████████████████████████████████████████████████████| 106/106 [00:21<00:00, 5.04it/s]
Loss: 0.6902545052316954 AUROC: 0.7967412321876445 AUPRC: 0.7205366976873793 Sensitivity: 0.7326290125965055 Specificity: 0.7134203168685928 Threshold: 0.14
best_model_auroc_test_auroc: 0.7967412321876445
best_model_auroc_test_auprc: 0.7205366976873793
When we complete our experiments, we will build comparison tables that compare a set of measures for each experiment performed. The full set of experiments and measures are listed below.
Note: each experiment will be repeated with the following time-to-IOH-event durations:
Note: the above list of experiments will be performed if there is sufficient time and gpu capability to complete that before the submission deadline. Should we experience any constraints on this front, we will reduce our experimental coverage to the following 4 core experiments that are necessary to measure the hypotheses included at the head of this report:
For additional details please review the "Planned Actions" in the Discussion section of this report.
[ TODO for final report - collect data for all measures listed above. ]
[ TODO for final report - generate ROC and PRC plots for each experiment ]
We are collecting a broad set of measures across each experiment in order to perform a comprehensive comparison of all measures listed across all comparable experiments executed in the original paper. However, our key experimental results will be focused on a subset of these results that address the main experiments defined at the beginning of this notebook.
The key experimental result measures will be as follows:
The following table is Table 3 from the original paper which presents the measured values for each signal combination across each of the four temporal predictive categories:
We have not yet completed the execution of the experiments necessary to determine our reproduced model performance in order determine whether our results are accurately representing those of the original paper. These details are expected to be included in the final report.
As of the draft submission, the reported evaluation measures of our model are too good to be true (all measures are 1.0). We suspect that there is data leakage in the dataset splitting process and will address this in time for the final report.
Our assessment is that this paper will be reproducible. The outstanding risk is that each experiment can take up to 7 hours to run on hardware within the team (i.e., 7h to run ~70 epochs on a desktop with AMD Ryzen 7 3800X 8-core CPU w/ RTX 2070 SUPER GPU and 32GB RAM). There are a total of 28 experiments (7 different combinations of signal inputs, 4 different time horizons for each combination). Should our team find it not possible to complete the necessary experiments across all of the experiments represented in Table 3 of our selected paper, we will reduce the number of experiments to focus solely on the ones directly related to our hypotheses described in the beginning of this notebook (i.e., reduce the number of combinations of interest to 4: ABP alone, ABP+EEG, ABP+ECG, ABP+ECG+EEG). This will result in a new total of 16 experiments to run.
Our proposal included a collection of potential ablations to be investigated:
Given the amount of time required to conduct each experiment, our team intends to choose only a small number of ablations from this set. Further, we only intend to perform ablation analysis against the best performing signal combination and time horizon from the reproduction experiments. In order words, we intend to perform ablation analysis against the following training combinations, and only against the models trained with data measured 3 minutes prior to an IOH event:
Time and GPU resource permitting, we will complete a broader range of experiments. For additional details, please see the section below titled "Plans for next phase".
Our team intends to address the manner in which the experimental results align with the published results in the paper in the final submission of this report. The amount of time required to complete model training and result analysis during the preparation of the Draft notebook was not sufficient to complete a large number of experiments.
The difficult aspect of the preparation of this draft involved the data preprocessing.
The most notable suggestion would be to correct the hyperparameters published in Supplemental Table 1. Specifically, the output size for residual blocks 11 and 12 for the ECG and ABP data sets was 496x6. This is a typo, and should read 469x6. This typo became apparent when operating the size down operation within Residual Block 11 and recognizing the tensor dimensions were misaligned.
Additionally, more explicit references to the signal quality index assessment tools should be added. Our team could not find a reference to the MATLAB source code as described in reference [3], and had to manually discover the GitHub profile for the lab of the corresponding author of reference [3] in order to find MATLAB source that corresponded to the metrics described therein.
Our team plans to accomplish the following goals in service of preparing the Final Report:
Walkthrough of the notebook, no need to make slides. We expect a well-timed, well-presented presentation. You should clearly explain what the original paper is about (what the general problem is, what the specific approach taken was, and what the results claimed were) and what you encountered when you attempted to reproduce the results. You should use the time given to you and not too much (or too little).
print(f'All done!')
All done!